Sep 19 05:40:52.993342 ip-10-0-131-202 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 05:40:56.291526 ip-10-0-131-202 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 05:40:56.291526 ip-10-0-131-202 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 05:40:56.291526 ip-10-0-131-202 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 05:40:56.291526 ip-10-0-131-202 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 05:40:56.291526 ip-10-0-131-202 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 05:40:56.292243 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.291621    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 05:40:56.295101 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295084    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:56.295101 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295101    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295105    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295108    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295112    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295115    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295118    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295121    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295128    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295131    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295134    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295137    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295139    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295142    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295145    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295148    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295151    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295154    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295156    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295159    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295161    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:56.295163 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295164    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295167    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295170    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295173    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295176    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295180    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295184    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295187    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295190    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295193    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295195    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295198    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295200    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295203    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295207    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295210    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295213    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295216    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295218    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:56.295634 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295221    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295224    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295226    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295229    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295231    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295233    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295236    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295238    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295241    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295243    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295246    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295248    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295250    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295253    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295255    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295259    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295262    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295265    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295269    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:56.296124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295272    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295275    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295278    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295281    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295283    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295286    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295288    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295291    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295293    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295296    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295304    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295307    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295310    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295313    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295315    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295319    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295322    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295325    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295328    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295330    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:56.296575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295333    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295335    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295338    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295340    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295343    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295345    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295348    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295351    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295355    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295358    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295360    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295363    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295365    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295368    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295370    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295373    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295375    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295378    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295380    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.295383    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:56.297075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296148    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296156    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296159    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296162    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296165    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296167    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296170    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296172    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296176    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296178    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296181    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296183    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296187    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296189    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296192    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296195    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296198    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296200    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296203    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296205    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:56.297575 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296210    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296212    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296215    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296217    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296220    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296223    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296225    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296228    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296230    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296233    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296235    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296238    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296241    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296243    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296248    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296252    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296256    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296259    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296262    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296264    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:56.298080 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296267    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296271    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296274    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296277    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296279    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296282    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296284    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296287    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296289    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296292    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296294    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296297    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296300    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296303    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296305    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296308    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296311    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296314    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296316    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296319    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:56.298576 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296321    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296324    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296326    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296329    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296331    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296334    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296336    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296339    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296341    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296343    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296346    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296349    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296351    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296353    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296356    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296359    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296364    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296368    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296370    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296372    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:56.299124 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296375    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296377    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296380    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296383    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296386    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296390    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296394    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296397    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296400    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296403    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296405    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296408    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296411    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296413    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296416    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296418    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296421    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296423    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296426    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:56.299623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296847    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296853    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296856    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296860    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296863    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296866    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296868    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296872    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296874    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296877    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296880    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296883    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296885    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296889    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296893    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296897    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296900    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296903    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296906    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:56.300117 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296909    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296911    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296914    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296917    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296919    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296922    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296924    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296927    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296929    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296932    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296934    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296937    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296939    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296942    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296944    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296947    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296949    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296952    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296954    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296957    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:56.300596 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296959    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296962    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296965    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296968    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296971    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296973    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296976    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296978    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296980    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296984    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296987    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296989    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296992    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296994    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296997    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.296999    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297002    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297004    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297007    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297010    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:56.301092 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297014    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297016    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297019    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297021    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297023    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297026    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297045    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297050    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297054    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297058    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297061    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297063    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297066    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297068    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297072    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297075    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297078    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297081    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297084    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297086    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:56.301605 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297089    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297091    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297097    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297099    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297103    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297106    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297109    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297112    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297115    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297118    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297121    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297124    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297127    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297129    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297132    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297135    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297138    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297141    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297144    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297146    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:56.302138 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297233    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297281    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297290    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297294    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297298    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297302    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297307    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297311    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297315    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297318    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297322    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297325    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297328    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297331    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297336    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297339    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297342    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297345    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297349    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297352    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297355    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297358    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297362    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297365    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 05:40:56.302612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297368    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297371    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297375    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297378    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297380    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297385    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297388    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297391    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297395    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297398    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297401    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297404    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297407    2581 flags.go:64] FLAG: --enable-server="true"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297410    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297415    2581 flags.go:64] FLAG: --event-burst="100"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297418    2581 flags.go:64] FLAG: --event-qps="50"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297422    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297425    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297428    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297432    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297435    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297438    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297442    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297445    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297448    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 05:40:56.303196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297451    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297454    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297456    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297459    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297462    2581 flags.go:64] FLAG: --feature-gates=""
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297466    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297469    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297472    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297475    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297478    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297481    2581 flags.go:64] FLAG: --help="false"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297484    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297488    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297491    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297494    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297497    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297501    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297503    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297506    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297509    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297512    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297515    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297518    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297521    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 19 05:40:56.303874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297524    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297527    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297530    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297533    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297536    2581 flags.go:64] FLAG: --lock-file=""
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297540    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297543    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297546    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297552    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297555    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297558    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297561    2581 flags.go:64] FLAG: --logging-format="text"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297564    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297567    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297570    2581 flags.go:64] FLAG: --manifest-url=""
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297573    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297577    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297580    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297584    2581 flags.go:64] FLAG: --max-pods="110"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297588    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297592    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297595    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297598    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297601    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297603    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 05:40:56.304512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297606    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297613    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297616    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297619    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297622    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297626    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297631    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297634    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297637    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297641    2581 flags.go:64] FLAG: --port="10250"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297644    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297647    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ca8d358edfcc6f3b"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297650    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297654    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297658    2581 flags.go:64] FLAG: --register-node="true"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297661    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297665    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297668    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297670    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297673    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297678    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297681    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297684    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297687    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297690    2581 flags.go:64] FLAG: --runonce="false"
Sep 19 05:40:56.305165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297693    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297697    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297700    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297704    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297707    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297710    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297713    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297717    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297720    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297722    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297725    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297728    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297731    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297734    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297737    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297742    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297746    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297749    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297753    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297756    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297760    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297763    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297766    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297769    2581 flags.go:64] FLAG: --v="2"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297773    2581 flags.go:64] FLAG: --version="false"
Sep 19 05:40:56.305877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297778    2581 flags.go:64] FLAG: --vmodule=""
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297781    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.297785    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297915    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297919    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297922    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297925    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297928    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297931    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297934    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297937    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297939    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297942    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297946    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297949    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297952    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297955    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297958    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297960    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:56.306917 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297963    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297966    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297969    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297972    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297974    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297977    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297980    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297982    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297986    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297989    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297991    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297994    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297996    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.297999    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298002    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298004    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298007    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298010    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298012    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298016    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:56.307523 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298019    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298022    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298025    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298042    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298045    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298048    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298051    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298054    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298057    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298059    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298062    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298064    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298067    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298069    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298072    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298075    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298077    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298079    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298082    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298085    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:56.308049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298090    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298093    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298096    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298099    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298101    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298104    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298106    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298109    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298112    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298114    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298117    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298120    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298122    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298124    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298128    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298131    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298133    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298136    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298138    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298141    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:56.308549 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298143    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298146    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298148    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298151    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298155    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298159    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298162    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298164    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298167    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298170    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298173    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298176    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298180    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298183    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298186    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298188    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298191    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298194    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298196    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298199    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:56.309045 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298201    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298204    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.298206    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.298731    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.307577    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.307592    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.307612    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.307617    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307672    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307677    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307681    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307684    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307704    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307708    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307711    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:56.309533 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307714    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307717    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307720    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307723    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307736    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307740    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307743    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307746    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307749    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307751    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307754    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307756    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307759    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307762    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307765    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307767    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307770    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307772    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307775    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:56.309909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307777    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307781    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307784    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307787    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307790    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307792    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307795    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307797    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307800    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307802    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307805    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307807    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307809    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307812    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307814    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307817    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307820    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307823    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307825    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307827    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:56.310514 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307830    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307833    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307835    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307837    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307840    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307843    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307845    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307848    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307850    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307853    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307855    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307858    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307860    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307863    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307866    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307869    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307872    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307874    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307877    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:56.311017 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307879    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307882    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307885    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307887    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307890    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307892    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307894    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307897    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307899    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307902    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307904    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307907    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307909    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307911    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307919    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307922    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307926    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307929    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307932    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:56.311540 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307935    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307938    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307940    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307943    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307946    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307948    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307952    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307954    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307957    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307960    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307962    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307965    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307968    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307970    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.307973    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:56.312012 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.307978    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308106    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308111    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308114    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308117    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308120    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308123    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308126    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308129    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308131    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308134    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308136    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308139    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308147    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308150    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308152    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308154    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308157    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308159    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308162    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:56.312411 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308164    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308168    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308171    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308174    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308177    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308179    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308182    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308184    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308187    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308190    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308192    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308195    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308197    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308199    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308202    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308204    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308207    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308209    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308212    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308214    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:56.312909 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308217    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308219    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308222    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308224    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308227    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308229    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308236    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308239    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308242    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308244    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308247    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308249    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308252    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308254    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308257    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308259    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308262    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308264    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308269    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:56.313421 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308272    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308276    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308279    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308281    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308284    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308287    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308289    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308292    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308295    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308298    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308301    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308304    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308307    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308309    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308312    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308315    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308318    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308320    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308323    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308325    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:56.313875 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308329    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308331    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308334    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308336    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308338    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308341    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308343    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308346    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308348    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308350    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308353    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308356    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308358    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308361    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308363    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308366    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308368    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308371    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308373    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308376    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:56.314454 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:56.308378    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.308383    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.308486    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.310312    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.311053    2581 server.go:1019] "Starting client certificate rotation"
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.311149    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.311770    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.312980    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 05:40:56.314930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.314747    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 05:40:56.318882 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.318869    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 19 05:40:56.320631 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.320619    2581 log.go:25] "Validated CRI v1 image API"
Sep 19 05:40:56.321570 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.321554    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 05:40:56.323785 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.323766    2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8c4422f0-8306-4991-a6bb-8fd0ca46b14e:/dev/nvme0n1p4 a5617cc2-463c-4330-8e42-f5e95864e5dd:/dev/nvme0n1p3]
Sep 19 05:40:56.323842 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.323784    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 05:40:56.329311 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.329181    2581 manager.go:216] Machine: {Timestamp:2026-09-19 05:40:56.32860052 +0000 UTC m=+1.939280494 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099104 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b7e988b02ccea0cb0b6c7e88ee10f SystemUUID:ec2b7e98-8b02-ccea-0cb0-b6c7e88ee10f BootID:86a0b794-5859-4ced-88a4-6deb3a7a4930 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c1:5b:98:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c1:5b:98:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:4b:d4:de:73:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 05:40:56.329311 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.329310    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 05:40:56.329414 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.329389    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 05:40:56.330932 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.330903    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 05:40:56.331112 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.330935    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-202.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 05:40:56.332138 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.332128    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 05:40:56.332170 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.332141    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 05:40:56.332170 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.332149    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 05:40:56.332170 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.332163    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 05:40:56.332620 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.332594    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 05:40:56.336524 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336508    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 05:40:56.336640 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336630    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 05:40:56.336680 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336664    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 05:40:56.336680 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336679    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 05:40:56.336774 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336690    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 05:40:56.336774 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336698    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 19 05:40:56.336774 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.336707    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 05:40:56.337167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.337153    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 05:40:56.337237 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.337180    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 05:40:56.341899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.341882    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 05:40:56.342824 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.342806    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 05:40:56.343768 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.343755    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 05:40:56.343809 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.343785    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 05:40:56.344896 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344885    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 05:40:56.344935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344901    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 05:40:56.344935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344907    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 05:40:56.344935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344912    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 05:40:56.344935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344920    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 05:40:56.344935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344926    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344938    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344943    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344950    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344955    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344964    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.344968    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.345044    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 05:40:56.345089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.345050    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 05:40:56.348639 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.348618    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-202.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 05:40:56.349111 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.349098    2581 server.go:1268] "Started kubelet"
Sep 19 05:40:56.349195 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.349160    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 05:40:56.349259 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.349179    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 05:40:56.349259 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.349252    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 05:40:56.350079 ip-10-0-131-202 systemd[1]: Started Kubernetes Kubelet.
Sep 19 05:40:56.350481 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.350287    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 05:40:56.350872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.350855    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 05:40:56.353504 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.353486    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 05:40:56.355142 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.355127    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 05:40:56.355272 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.355166    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 05:40:56.355272 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.355229    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.355588 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.355482    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 05:40:56.355718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.355705    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 05:40:56.355866 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.355849    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 05:40:56.355938 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.355868    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 05:40:56.355991 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.355938    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 05:40:56.356055 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356012    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 05:40:56.356055 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356021    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 05:40:56.356363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356350    2581 factory.go:153] Registering CRI-O factory
Sep 19 05:40:56.356363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356365    2581 factory.go:223] Registration of the crio container factory successfully
Sep 19 05:40:56.356473 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356437    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 05:40:56.356473 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356448    2581 factory.go:55] Registering systemd factory
Sep 19 05:40:56.356473 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356456    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 19 05:40:56.356605 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356486    2581 factory.go:103] Registering Raw factory
Sep 19 05:40:56.356605 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.356503    2581 manager.go:1196] Started watching for new ooms in manager
Sep 19 05:40:56.357060 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.357025    2581 manager.go:349] Starting recovery of all containers
Sep 19 05:40:56.377252 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.377226    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g6d8j"
Sep 19 05:40:56.379607 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.379567    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g6d8j"
Sep 19 05:40:56.381337 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.381300    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 05:40:56.381537 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.381469    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 05:40:56.383431 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.383396    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 05:40:56.385232 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.384462    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-202.ec2.internal.18d6a264f0f084ca  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-202.ec2.internal,UID:ip-10-0-131-202.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-202.ec2.internal,},FirstTimestamp:2026-09-19 05:40:56.349074634 +0000 UTC m=+1.959754590,LastTimestamp:2026-09-19 05:40:56.349074634 +0000 UTC m=+1.959754590,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-202.ec2.internal,}"
Sep 19 05:40:56.388067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.388053    2581 manager.go:354] Recovery completed
Sep 19 05:40:56.392697 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.392683    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.394476 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.394460    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.394533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.394494    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.394533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.394508    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.395027 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.395011    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 05:40:56.395132 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.395047    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 05:40:56.395132 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.395065    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 05:40:56.395132 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.395097    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.396025 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.396011    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.396025 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.396052    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.396025 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.396067    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.397124 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.397112    2581 policy_none.go:49] "None policy: Start"
Sep 19 05:40:56.397170 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.397128    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 05:40:56.397170 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.397138    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 05:40:56.399076 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.399066    2581 policy_none.go:47] "Start"
Sep 19 05:40:56.432751 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.432729    2581 manager.go:337] "Starting Device Plugin manager"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.432795    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.432807    2581 server.go:87] "Starting device plugin registration server"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.433176    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.433198    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.433329    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.433403    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.433414    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.433949    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 05:40:56.453728 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.434004    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.470437 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.470407    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 05:40:56.470541 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.470456    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 05:40:56.470541 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.470478    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 05:40:56.470541 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.470518    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 05:40:56.472459 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.472437    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:56.533703 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.533666    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.534752 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.534728    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.534879 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.534771    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.534879 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.534784    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.534879 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.534808    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.540870 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.540851    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.540976 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.540879    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-202.ec2.internal\": node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.560898 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.560837    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.570812 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.570784    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"]
Sep 19 05:40:56.570921 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.570847    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.571782 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.571767    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.571850 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.571795    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.571850 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.571804    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.573984 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.573973    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.574118 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.574173 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574164    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.574721 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574703    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.574721 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574702    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.574849 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574736    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.574849 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574738    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.574849 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574746    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.574849 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.574753    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.576932 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.576915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.577009 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.576939    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:56.577613 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.577593    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:56.577687 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.577634    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:56.577687 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.577651    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:56.601001 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.600977    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-202.ec2.internal\" not found" node="ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.606145 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.606125    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-202.ec2.internal\" not found" node="ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.657058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.657012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ece54a7e142bdf40d1f2f69c78afd8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal\" (UID: \"3ece54a7e142bdf40d1f2f69c78afd8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.657058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.657059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ece54a7e142bdf40d1f2f69c78afd8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal\" (UID: \"3ece54a7e142bdf40d1f2f69c78afd8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.657253 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.657125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c990e3bd2b767d7e89440c7596b76213-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.ec2.internal\" (UID: \"c990e3bd2b767d7e89440c7596b76213\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.661111 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.661094    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.758239 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.758197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ece54a7e142bdf40d1f2f69c78afd8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal\" (UID: \"3ece54a7e142bdf40d1f2f69c78afd8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.758239 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.758236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c990e3bd2b767d7e89440c7596b76213-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.ec2.internal\" (UID: \"c990e3bd2b767d7e89440c7596b76213\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.758434 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.758254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ece54a7e142bdf40d1f2f69c78afd8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal\" (UID: \"3ece54a7e142bdf40d1f2f69c78afd8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.758434 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.758312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ece54a7e142bdf40d1f2f69c78afd8b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal\" (UID: \"3ece54a7e142bdf40d1f2f69c78afd8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.758434 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.758323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ece54a7e142bdf40d1f2f69c78afd8b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal\" (UID: \"3ece54a7e142bdf40d1f2f69c78afd8b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.758434 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.758330    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c990e3bd2b767d7e89440c7596b76213-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.ec2.internal\" (UID: \"c990e3bd2b767d7e89440c7596b76213\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.761299 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.761280    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.862163 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.862098    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:56.904308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.904268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.910158 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:56.909494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:56.963218 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:56.963190    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:57.063699 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:57.063672    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:57.164219 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:57.164146    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:57.198426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.198404    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:57.264490 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:57.264462    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:57.311649 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.311618    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 05:40:57.312125 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.311778    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 05:40:57.312125 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.311789    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 05:40:57.365046 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:57.365015    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-202.ec2.internal\" not found"
Sep 19 05:40:57.386661 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.386623    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 05:35:56 +0000 UTC" deadline="2028-04-28 03:37:08.710613754 +0000 UTC"
Sep 19 05:40:57.386661 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.386658    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14085h56m11.323958975s"
Sep 19 05:40:57.431161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.430971    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:57.456409 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.456372    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:57.467678 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.467658    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 05:40:57.469094 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.469079    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal"
Sep 19 05:40:57.475652 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.475636    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 05:40:57.581273 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:57.581243    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc990e3bd2b767d7e89440c7596b76213.slice/crio-be591c45d50d89193560526cba1a0f8cd0dcae11c08d3c0647e990fa9c188fed WatchSource:0}: Error finding container be591c45d50d89193560526cba1a0f8cd0dcae11c08d3c0647e990fa9c188fed: Status 404 returned error can't find the container with id be591c45d50d89193560526cba1a0f8cd0dcae11c08d3c0647e990fa9c188fed
Sep 19 05:40:57.581573 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:57.581556    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ece54a7e142bdf40d1f2f69c78afd8b.slice/crio-9bfcdc21e76d718f15b7564c07b69e1483d8855457bd3deda15f09bee47ddcc1 WatchSource:0}: Error finding container 9bfcdc21e76d718f15b7564c07b69e1483d8855457bd3deda15f09bee47ddcc1: Status 404 returned error can't find the container with id 9bfcdc21e76d718f15b7564c07b69e1483d8855457bd3deda15f09bee47ddcc1
Sep 19 05:40:57.586449 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.586430    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:40:57.814267 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:57.814234    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:58.337488 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.337458    2581 apiserver.go:52] "Watching apiserver"
Sep 19 05:40:58.345553 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.345525    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 05:40:58.345963 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.345935    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-qp5nd","openshift-network-operator/iptables-alerter-5hm7m","kube-system/global-pull-secret-syncer-hq78s","kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal","openshift-cluster-node-tuning-operator/tuned-2mcq9","openshift-dns/node-resolver-pcmxb","openshift-multus/multus-additional-cni-plugins-xb7qg","openshift-multus/network-metrics-daemon-cnxxg","openshift-network-diagnostics/network-check-target-52bg8","openshift-ovn-kubernetes/ovnkube-node-r9592","kube-system/konnectivity-agent-8gb7c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x","openshift-image-registry/node-ca-wrrj9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal"]
Sep 19 05:40:58.348547 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.348522    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.350745 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.350714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.351872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.351854    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.351967 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.351882    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 05:40:58.352491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352169    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 05:40:58.352491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352290    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c95xg\""
Sep 19 05:40:58.352491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352342    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 05:40:58.352491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352375    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 05:40:58.352491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352375    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.352942 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.353047 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.352995    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.353047 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.352996    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:40:58.354323 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.354305    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 05:40:58.354550 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.354533    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.354708 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.354690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qpkfm\""
Sep 19 05:40:58.355612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.355582    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 05:40:58.355788 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.355630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.355788 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.355685    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:40:58.357792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.357769    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.360587 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.360560    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.360587 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.360575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rc6pn\""
Sep 19 05:40:58.360725 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.360615    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.360725 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.360617    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.362794 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.362772    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:58.363230 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.363121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:40:58.363230 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.363195    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:40:58.363976 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.363956    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.364162 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.364146    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.364317 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.364302    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-28qr6\""
Sep 19 05:40:58.365476 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.365456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.366098 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-run-netns\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366180 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-var-lib-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366180 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-log-socket\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366279 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysctl-conf\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.366279 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-systemd\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.366279 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-host\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.366279 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-tmp\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.366471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-kubelet\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-run-ovn-kubernetes\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-env-overrides\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-ovnkube-script-lib\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-kubernetes\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-var-lib-kubelet\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-node-log\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b39f1022-4cf2-4513-a0fb-92216b5a67e3-serviceca\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhzcl\" (UniqueName: \"kubernetes.io/projected/b39f1022-4cf2-4513-a0fb-92216b5a67e3-kube-api-access-nhzcl\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/78f960ee-50d2-40db-a972-81da4ef0d346-dbus\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-systemd-units\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tchhm\" (UniqueName: \"kubernetes.io/projected/95225afd-e021-41c3-8fcb-b99d36b67473-kube-api-access-tchhm\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b39f1022-4cf2-4513-a0fb-92216b5a67e3-host\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.366802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/78f960ee-50d2-40db-a972-81da4ef0d346-kubelet-config\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-modprobe-d\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-etc-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-ovn\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.366960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-tuned\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsw29\" (UniqueName: \"kubernetes.io/projected/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-kube-api-access-wsw29\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-cni-bin\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95225afd-e021-41c3-8fcb-b99d36b67473-ovn-node-metrics-cert\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysconfig\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysctl-d\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-run\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-systemd\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-cni-netd\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-ovnkube-config\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.367411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-lib-modules\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.368023 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-slash\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.368023 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqvx8\" (UniqueName: \"kubernetes.io/projected/6c42040f-8e4f-4d40-b199-58def6a92b99-kube-api-access-fqvx8\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.368023 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.367397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-sys\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.369556 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.368647    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.369556 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.368904    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-db2d6\""
Sep 19 05:40:58.369556 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.368933    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.369556 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.368952    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 05:40:58.369556 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.369302    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.370057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.369850    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 05:40:58.370057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.369859    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 05:40:58.370824 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.370790    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 05:40:58.371951 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.371929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.372540 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.372520    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.372627 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.372562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lmx88\""
Sep 19 05:40:58.372627 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.372574    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.372759 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.372745    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 05:40:58.374813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.374793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.376145 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.376128    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 05:40:58.376427 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.376407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7wpl7\""
Sep 19 05:40:58.376914 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.376896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.377429 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.377413    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 05:40:58.377519 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.377468    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 05:40:58.378161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.378140    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sbt5m\""
Sep 19 05:40:58.383058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.382381    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wdx4s\""
Sep 19 05:40:58.383058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.382502    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 05:40:58.383058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.382560    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 05:40:58.383058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.382738    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 05:40:58.409396 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.409376    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mp28x"
Sep 19 05:40:58.420346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.420324    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mp28x"
Sep 19 05:40:58.460304 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.460276    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 05:40:58.467630 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-kubelet\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.467766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.467766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-ovnkube-script-lib\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.467766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-kubernetes\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.467766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da9d45fc-33f1-4fad-b8ba-35bedabba559-hosts-file\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.467766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-node-log\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/78f960ee-50d2-40db-a972-81da4ef0d346-dbus\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e349d24e-28fc-4824-adad-292ac8dee6e9-konnectivity-ca\") pod \"konnectivity-agent-8gb7c\" (UID: \"e349d24e-28fc-4824-adad-292ac8dee6e9\") " pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-systemd-units\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-cnibin\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-kubelet\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-hostroot\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-sys-fs\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.467990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-etc-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-ovn\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468162    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-kubernetes\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-kubelet\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-etc-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-node-log\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-systemd-units\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.468309    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.468393    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:58.968354545 +0000 UTC m=+4.579034502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-ovn\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.468588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/78f960ee-50d2-40db-a972-81da4ef0d346-dbus\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-ovnkube-script-lib\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-tuned\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.468923    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.468946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da9d45fc-33f1-4fad-b8ba-35bedabba559-tmp-dir\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.468979    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:58.968958794 +0000 UTC m=+4.579638751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-system-cni-dir\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtznt\" (UniqueName: \"kubernetes.io/projected/95f848fd-2aef-41c3-92b7-c622d84c6c85-kube-api-access-mtznt\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-cnibin\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469099    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95225afd-e021-41c3-8fcb-b99d36b67473-ovn-node-metrics-cert\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysctl-d\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469102    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 05:40:58.469183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-run\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysctl-d\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-run\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95f848fd-2aef-41c3-92b7-c622d84c6c85-host-slash\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-socket-dir-parent\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-systemd\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469575    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-run-systemd\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-ovnkube-config\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-lib-modules\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-conf-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98470b56-3241-4618-9bf4-d6578d14c33d-multus-daemon-config\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-sys\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98470b56-3241-4618-9bf4-d6578d14c33d-cni-binary-copy\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-var-lib-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-systemd\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.469854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-system-cni-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-run-ovn-kubernetes\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-env-overrides\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-var-lib-kubelet\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.469996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95f848fd-2aef-41c3-92b7-c622d84c6c85-iptables-alerter-script\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-ovnkube-config\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljn6r\" (UniqueName: \"kubernetes.io/projected/98470b56-3241-4618-9bf4-d6578d14c33d-kube-api-access-ljn6r\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b39f1022-4cf2-4513-a0fb-92216b5a67e3-serviceca\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-systemd\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-lib-modules\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-sys\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-var-lib-openvswitch\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhzcl\" (UniqueName: \"kubernetes.io/projected/b39f1022-4cf2-4513-a0fb-92216b5a67e3-kube-api-access-nhzcl\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tchhm\" (UniqueName: \"kubernetes.io/projected/95225afd-e021-41c3-8fcb-b99d36b67473-kube-api-access-tchhm\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470403    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95225afd-e021-41c3-8fcb-b99d36b67473-env-overrides\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.470552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b39f1022-4cf2-4513-a0fb-92216b5a67e3-host\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-var-lib-kubelet\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/78f960ee-50d2-40db-a972-81da4ef0d346-kubelet-config\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-modprobe-d\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-multus-certs\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-device-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsw29\" (UniqueName: \"kubernetes.io/projected/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-kube-api-access-wsw29\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-cni-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-etc-selinux\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-cni-bin\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysconfig\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b39f1022-4cf2-4513-a0fb-92216b5a67e3-host\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-cni-multus\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysconfig\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtqms\" (UniqueName: \"kubernetes.io/projected/701d115e-2e55-4744-9999-6f5f09155082-kube-api-access-mtqms\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-cni-netd\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.471083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e349d24e-28fc-4824-adad-292ac8dee6e9-agent-certs\") pod \"konnectivity-agent-8gb7c\" (UID: \"e349d24e-28fc-4824-adad-292ac8dee6e9\") " pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.471789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-registration-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.471789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-slash\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.471789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.470998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqvx8\" (UniqueName: \"kubernetes.io/projected/6c42040f-8e4f-4d40-b199-58def6a92b99-kube-api-access-fqvx8\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.471789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kvvr\" (UniqueName: \"kubernetes.io/projected/da9d45fc-33f1-4fad-b8ba-35bedabba559-kube-api-access-8kvvr\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.471789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46g7\" (UniqueName: \"kubernetes.io/projected/2b530891-1fbd-4286-a117-e437388721eb-kube-api-access-r46g7\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.473853 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-k8s-cni-cncf-io\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.473859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-netns\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.473887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-socket-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471223    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b39f1022-4cf2-4513-a0fb-92216b5a67e3-serviceca\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.473916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-run-netns\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-cni-bin\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471403    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-cni-netd\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-slash\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.473971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.472815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-tuned\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.472963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95225afd-e021-41c3-8fcb-b99d36b67473-ovn-node-metrics-cert\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.473982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-log-socket\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471218    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-run-ovn-kubernetes\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-modprobe-d\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.473951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-log-socket\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.471257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/78f960ee-50d2-40db-a972-81da4ef0d346-kubelet-config\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysctl-conf\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-host\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95225afd-e021-41c3-8fcb-b99d36b67473-host-run-netns\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-tmp\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-cni-bin\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-host\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-etc-kubernetes\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.474352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.474329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-etc-sysctl-conf\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.477056 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.476969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-tmp\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.478245 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.478199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal" event={"ID":"c990e3bd2b767d7e89440c7596b76213","Type":"ContainerStarted","Data":"be591c45d50d89193560526cba1a0f8cd0dcae11c08d3c0647e990fa9c188fed"}
Sep 19 05:40:58.480154 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.480129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal" event={"ID":"3ece54a7e142bdf40d1f2f69c78afd8b","Type":"ContainerStarted","Data":"9bfcdc21e76d718f15b7564c07b69e1483d8855457bd3deda15f09bee47ddcc1"}
Sep 19 05:40:58.481027 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.480997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tchhm\" (UniqueName: \"kubernetes.io/projected/95225afd-e021-41c3-8fcb-b99d36b67473-kube-api-access-tchhm\") pod \"ovnkube-node-r9592\" (UID: \"95225afd-e021-41c3-8fcb-b99d36b67473\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.481140 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.481063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhzcl\" (UniqueName: \"kubernetes.io/projected/b39f1022-4cf2-4513-a0fb-92216b5a67e3-kube-api-access-nhzcl\") pod \"node-ca-wrrj9\" (UID: \"b39f1022-4cf2-4513-a0fb-92216b5a67e3\") " pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.481497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.481479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqvx8\" (UniqueName: \"kubernetes.io/projected/6c42040f-8e4f-4d40-b199-58def6a92b99-kube-api-access-fqvx8\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.483162 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.483139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsw29\" (UniqueName: \"kubernetes.io/projected/cbffbbbd-ec33-42ab-b0e6-65fd2297af7b-kube-api-access-wsw29\") pod \"tuned-2mcq9\" (UID: \"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b\") " pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.574500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.574500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-system-cni-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95f848fd-2aef-41c3-92b7-c622d84c6c85-iptables-alerter-script\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljn6r\" (UniqueName: \"kubernetes.io/projected/98470b56-3241-4618-9bf4-d6578d14c33d-kube-api-access-ljn6r\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-multus-certs\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-system-cni-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-device-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-cni-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-device-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-etc-selinux\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.574734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574726    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-cni-multus\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtqms\" (UniqueName: \"kubernetes.io/projected/701d115e-2e55-4744-9999-6f5f09155082-kube-api-access-mtqms\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e349d24e-28fc-4824-adad-292ac8dee6e9-agent-certs\") pod \"konnectivity-agent-8gb7c\" (UID: \"e349d24e-28fc-4824-adad-292ac8dee6e9\") " pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-registration-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kvvr\" (UniqueName: \"kubernetes.io/projected/da9d45fc-33f1-4fad-b8ba-35bedabba559-kube-api-access-8kvvr\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46g7\" (UniqueName: \"kubernetes.io/projected/2b530891-1fbd-4286-a117-e437388721eb-kube-api-access-r46g7\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-k8s-cni-cncf-io\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-netns\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-socket-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.574994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-cni-bin\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-etc-kubernetes\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da9d45fc-33f1-4fad-b8ba-35bedabba559-hosts-file\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e349d24e-28fc-4824-adad-292ac8dee6e9-konnectivity-ca\") pod \"konnectivity-agent-8gb7c\" (UID: \"e349d24e-28fc-4824-adad-292ac8dee6e9\") " pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da9d45fc-33f1-4fad-b8ba-35bedabba559-hosts-file\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-cnibin\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.575229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-kubelet\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-hostroot\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-cni-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-sys-fs\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da9d45fc-33f1-4fad-b8ba-35bedabba559-tmp-dir\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-system-cni-dir\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575248    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95f848fd-2aef-41c3-92b7-c622d84c6c85-iptables-alerter-script\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtznt\" (UniqueName: \"kubernetes.io/projected/95f848fd-2aef-41c3-92b7-c622d84c6c85-kube-api-access-mtznt\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575284    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-etc-selinux\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-cnibin\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95f848fd-2aef-41c3-92b7-c622d84c6c85-host-slash\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-socket-dir-parent\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-conf-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98470b56-3241-4618-9bf4-d6578d14c33d-multus-daemon-config\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98470b56-3241-4618-9bf4-d6578d14c33d-cni-binary-copy\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-cnibin\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-cni-multus\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-cni-bin\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575619    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-etc-kubernetes\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e349d24e-28fc-4824-adad-292ac8dee6e9-konnectivity-ca\") pod \"konnectivity-agent-8gb7c\" (UID: \"e349d24e-28fc-4824-adad-292ac8dee6e9\") " pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-cnibin\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-netns\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-k8s-cni-cncf-io\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-registration-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-socket-dir\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-hostroot\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-socket-dir-parent\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b530891-1fbd-4286-a117-e437388721eb-system-cni-dir\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/701d115e-2e55-4744-9999-6f5f09155082-sys-fs\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.576729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.575458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-run-multus-certs\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b530891-1fbd-4286-a117-e437388721eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da9d45fc-33f1-4fad-b8ba-35bedabba559-tmp-dir\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576499    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-host-var-lib-kubelet\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95f848fd-2aef-41c3-92b7-c622d84c6c85-host-slash\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98470b56-3241-4618-9bf4-d6578d14c33d-multus-conf-dir\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98470b56-3241-4618-9bf4-d6578d14c33d-cni-binary-copy\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.577500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.576887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98470b56-3241-4618-9bf4-d6578d14c33d-multus-daemon-config\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.578995 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.578968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e349d24e-28fc-4824-adad-292ac8dee6e9-agent-certs\") pod \"konnectivity-agent-8gb7c\" (UID: \"e349d24e-28fc-4824-adad-292ac8dee6e9\") " pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.584047 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.584009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtznt\" (UniqueName: \"kubernetes.io/projected/95f848fd-2aef-41c3-92b7-c622d84c6c85-kube-api-access-mtznt\") pod \"iptables-alerter-5hm7m\" (UID: \"95f848fd-2aef-41c3-92b7-c622d84c6c85\") " pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.586199 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.586178    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:58.586199 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.586201    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:58.586357 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.586211    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:58.586357 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.586263    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:40:59.086250864 +0000 UTC m=+4.696930818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:58.588538 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.588489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljn6r\" (UniqueName: \"kubernetes.io/projected/98470b56-3241-4618-9bf4-d6578d14c33d-kube-api-access-ljn6r\") pod \"multus-qp5nd\" (UID: \"98470b56-3241-4618-9bf4-d6578d14c33d\") " pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.589145 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.589104    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kvvr\" (UniqueName: \"kubernetes.io/projected/da9d45fc-33f1-4fad-b8ba-35bedabba559-kube-api-access-8kvvr\") pod \"node-resolver-pcmxb\" (UID: \"da9d45fc-33f1-4fad-b8ba-35bedabba559\") " pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.589227 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.589198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46g7\" (UniqueName: \"kubernetes.io/projected/2b530891-1fbd-4286-a117-e437388721eb-kube-api-access-r46g7\") pod \"multus-additional-cni-plugins-xb7qg\" (UID: \"2b530891-1fbd-4286-a117-e437388721eb\") " pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.589400 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.589380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtqms\" (UniqueName: \"kubernetes.io/projected/701d115e-2e55-4744-9999-6f5f09155082-kube-api-access-mtqms\") pod \"aws-ebs-csi-driver-node-m692x\" (UID: \"701d115e-2e55-4744-9999-6f5f09155082\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.663392 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.663342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:40:58.671194 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.671173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wrrj9"
Sep 19 05:40:58.679335 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.679304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mcq9"
Sep 19 05:40:58.686854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.686834    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pcmxb"
Sep 19 05:40:58.695502 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.695485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xb7qg"
Sep 19 05:40:58.710050 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.710016    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hm7m"
Sep 19 05:40:58.716740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.716719    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qp5nd"
Sep 19 05:40:58.727289 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.727267    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:40:58.731885 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.731867    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x"
Sep 19 05:40:58.978689 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.978603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:58.978689 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:58.978659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:58.978908 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.978745    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:58.978908 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.978783    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:58.978908 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.978843    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:59.978823934 +0000 UTC m=+5.589503882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:58.978908 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:58.978864    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:59.978853978 +0000 UTC m=+5.589533938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:59.179533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:59.179502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:40:59.179683 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.179635    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:59.179683 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.179651    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:59.179683 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.179660    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:59.179796 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.179709    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:00.179696495 +0000 UTC m=+5.790376437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:59.375881 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:59.375849    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:59.424581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:59.424510    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 05:35:58 +0000 UTC" deadline="2028-03-03 21:12:49.423855841 +0000 UTC"
Sep 19 05:40:59.424581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:59.424577    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12759h31m49.999283545s"
Sep 19 05:40:59.535155 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.535127    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9d45fc_33f1_4fad_b8ba_35bedabba559.slice/crio-e10102db882fea25add3a62e9e8eea34dfd32beb971f36a3bbda00078d67633a WatchSource:0}: Error finding container e10102db882fea25add3a62e9e8eea34dfd32beb971f36a3bbda00078d67633a: Status 404 returned error can't find the container with id e10102db882fea25add3a62e9e8eea34dfd32beb971f36a3bbda00078d67633a
Sep 19 05:40:59.537217 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.537055    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbffbbbd_ec33_42ab_b0e6_65fd2297af7b.slice/crio-c42637ab71e98184b15834181bd5e724008f9697e9dd15c20de71fddad87ffba WatchSource:0}: Error finding container c42637ab71e98184b15834181bd5e724008f9697e9dd15c20de71fddad87ffba: Status 404 returned error can't find the container with id c42637ab71e98184b15834181bd5e724008f9697e9dd15c20de71fddad87ffba
Sep 19 05:40:59.540863 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.540792    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b530891_1fbd_4286_a117_e437388721eb.slice/crio-89c44bab2e2198787651f11051382f88c291ddeddda27ac3326133f80f210ab8 WatchSource:0}: Error finding container 89c44bab2e2198787651f11051382f88c291ddeddda27ac3326133f80f210ab8: Status 404 returned error can't find the container with id 89c44bab2e2198787651f11051382f88c291ddeddda27ac3326133f80f210ab8
Sep 19 05:40:59.543114 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.543088    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95225afd_e021_41c3_8fcb_b99d36b67473.slice/crio-d4af545ac43a9b278c4a9196212b9ce182b26e9e0cb6d49803ef34a415defddb WatchSource:0}: Error finding container d4af545ac43a9b278c4a9196212b9ce182b26e9e0cb6d49803ef34a415defddb: Status 404 returned error can't find the container with id d4af545ac43a9b278c4a9196212b9ce182b26e9e0cb6d49803ef34a415defddb
Sep 19 05:40:59.543501 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.543477    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39f1022_4cf2_4513_a0fb_92216b5a67e3.slice/crio-e6e8fd93b84302174c24649e421fd68aa9bde4f5e1ae96008c6f55be8cd0f809 WatchSource:0}: Error finding container e6e8fd93b84302174c24649e421fd68aa9bde4f5e1ae96008c6f55be8cd0f809: Status 404 returned error can't find the container with id e6e8fd93b84302174c24649e421fd68aa9bde4f5e1ae96008c6f55be8cd0f809
Sep 19 05:40:59.545248 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.545065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98470b56_3241_4618_9bf4_d6578d14c33d.slice/crio-82f2e1c6341ffbab45aa22dcf01045129014a4ffa987d761ee024ce636a98a3e WatchSource:0}: Error finding container 82f2e1c6341ffbab45aa22dcf01045129014a4ffa987d761ee024ce636a98a3e: Status 404 returned error can't find the container with id 82f2e1c6341ffbab45aa22dcf01045129014a4ffa987d761ee024ce636a98a3e
Sep 19 05:40:59.546234 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.546103    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f848fd_2aef_41c3_92b7_c622d84c6c85.slice/crio-4b67c379b3dc06bbf1d9e5241e966627165d2db34763b9905e3b091d89dd34f6 WatchSource:0}: Error finding container 4b67c379b3dc06bbf1d9e5241e966627165d2db34763b9905e3b091d89dd34f6: Status 404 returned error can't find the container with id 4b67c379b3dc06bbf1d9e5241e966627165d2db34763b9905e3b091d89dd34f6
Sep 19 05:40:59.546899 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.546703    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode349d24e_28fc_4824_adad_292ac8dee6e9.slice/crio-3d6887968387935d8ed8ad159b28a6ade27a434d21f5ef4be8cc0a312b31f670 WatchSource:0}: Error finding container 3d6887968387935d8ed8ad159b28a6ade27a434d21f5ef4be8cc0a312b31f670: Status 404 returned error can't find the container with id 3d6887968387935d8ed8ad159b28a6ade27a434d21f5ef4be8cc0a312b31f670
Sep 19 05:40:59.547996 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:40:59.547864    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701d115e_2e55_4744_9999_6f5f09155082.slice/crio-1d1fd4795ee5f9a3d1f3ebd3bbf8ff256293e120a23709ff886501f3e923b6c8 WatchSource:0}: Error finding container 1d1fd4795ee5f9a3d1f3ebd3bbf8ff256293e120a23709ff886501f3e923b6c8: Status 404 returned error can't find the container with id 1d1fd4795ee5f9a3d1f3ebd3bbf8ff256293e120a23709ff886501f3e923b6c8
Sep 19 05:40:59.986469 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:59.986385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:40:59.986469 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:40:59.986442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:40:59.986702 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.986576    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:59.986702 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.986624    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:01.986611151 +0000 UTC m=+7.597291098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:59.987003 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.986975    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:59.987149 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:40:59.987050    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:01.987015021 +0000 UTC m=+7.597694974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:00.188758 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.188194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:00.188758 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.188379    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:00.188758 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.188395    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:00.188758 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.188403    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:00.188758 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.188456    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:02.188434565 +0000 UTC m=+7.799114506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:00.425358 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.425312    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 05:35:58 +0000 UTC" deadline="2028-05-05 04:53:29.173534552 +0000 UTC"
Sep 19 05:41:00.425358 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.425349    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14255h12m28.748189939s"
Sep 19 05:41:00.472903 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.472169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:00.472903 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.472298    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:00.472903 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.472689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:00.472903 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.472780    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:00.473582 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.473340    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:00.473582 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:00.473444    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:00.497634 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.497575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wrrj9" event={"ID":"b39f1022-4cf2-4513-a0fb-92216b5a67e3","Type":"ContainerStarted","Data":"e6e8fd93b84302174c24649e421fd68aa9bde4f5e1ae96008c6f55be8cd0f809"}
Sep 19 05:41:00.500453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.500423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qp5nd" event={"ID":"98470b56-3241-4618-9bf4-d6578d14c33d","Type":"ContainerStarted","Data":"82f2e1c6341ffbab45aa22dcf01045129014a4ffa987d761ee024ce636a98a3e"}
Sep 19 05:41:00.508562 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.508531    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"d4af545ac43a9b278c4a9196212b9ce182b26e9e0cb6d49803ef34a415defddb"}
Sep 19 05:41:00.512480 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.512452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mcq9" event={"ID":"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b","Type":"ContainerStarted","Data":"c42637ab71e98184b15834181bd5e724008f9697e9dd15c20de71fddad87ffba"}
Sep 19 05:41:00.519996 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.519966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x" event={"ID":"701d115e-2e55-4744-9999-6f5f09155082","Type":"ContainerStarted","Data":"1d1fd4795ee5f9a3d1f3ebd3bbf8ff256293e120a23709ff886501f3e923b6c8"}
Sep 19 05:41:00.522170 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.522143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8gb7c" event={"ID":"e349d24e-28fc-4824-adad-292ac8dee6e9","Type":"ContainerStarted","Data":"3d6887968387935d8ed8ad159b28a6ade27a434d21f5ef4be8cc0a312b31f670"}
Sep 19 05:41:00.529336 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.529310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerStarted","Data":"89c44bab2e2198787651f11051382f88c291ddeddda27ac3326133f80f210ab8"}
Sep 19 05:41:00.538022 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.537950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pcmxb" event={"ID":"da9d45fc-33f1-4fad-b8ba-35bedabba559","Type":"ContainerStarted","Data":"e10102db882fea25add3a62e9e8eea34dfd32beb971f36a3bbda00078d67633a"}
Sep 19 05:41:00.545632 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.545575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal" event={"ID":"c990e3bd2b767d7e89440c7596b76213","Type":"ContainerStarted","Data":"4d311e38b58f1fd43bf887edcf89bd309c63a277019c787009f29cedc7ea8382"}
Sep 19 05:41:00.552188 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.552159    2581 generic.go:356] "Generic (PLEG): container finished" podID="3ece54a7e142bdf40d1f2f69c78afd8b" containerID="da0191155eb20888618f704d488201866369346cf3676baffb91884210b45b8b" exitCode=0
Sep 19 05:41:00.552296 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.552242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal" event={"ID":"3ece54a7e142bdf40d1f2f69c78afd8b","Type":"ContainerDied","Data":"da0191155eb20888618f704d488201866369346cf3676baffb91884210b45b8b"}
Sep 19 05:41:00.561700 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.561518    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.ec2.internal" podStartSLOduration=3.561505384 podStartE2EDuration="3.561505384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:00.561125167 +0000 UTC m=+6.171805148" watchObservedRunningTime="2026-09-19 05:41:00.561505384 +0000 UTC m=+6.172185347"
Sep 19 05:41:00.562686 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:00.562658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hm7m" event={"ID":"95f848fd-2aef-41c3-92b7-c622d84c6c85","Type":"ContainerStarted","Data":"4b67c379b3dc06bbf1d9e5241e966627165d2db34763b9905e3b091d89dd34f6"}
Sep 19 05:41:01.588028 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:01.587768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal" event={"ID":"3ece54a7e142bdf40d1f2f69c78afd8b","Type":"ContainerStarted","Data":"543910f6106ffa8d12db0fc23a1efc3e6c4ec5a7cf68fdb56c3ebcf9b1628593"}
Sep 19 05:41:02.000869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:02.000559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:02.000869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:02.000613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:02.000869 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.000726    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:02.000869 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.000753    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:02.000869 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.000813    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:06.000784218 +0000 UTC m=+11.611464177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:02.000869 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.000834    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:06.000823721 +0000 UTC m=+11.611503678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:02.203501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:02.202872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:02.203501 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.203054    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:02.203501 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.203075    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:02.203501 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.203087    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:02.203501 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.203145    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:06.203126918 +0000 UTC m=+11.813806865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:02.473352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:02.473317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:02.473515 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.473455    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:02.473872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:02.473852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:02.473977 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.473957    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:02.474064 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:02.474049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:02.474165 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:02.474135    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:04.471884 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:04.471160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:04.471884 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:04.471297    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:04.471884 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:04.471709    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:04.471884 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:04.471833    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:04.472552 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:04.471986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:04.472552 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:04.472100    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:06.035414 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:06.035374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:06.035896 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:06.035431    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:06.035896 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.035589    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:06.035896 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.035654    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:14.035633735 +0000 UTC m=+19.646313681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:06.036127 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.036105    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:06.036215 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.036167    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:14.036150786 +0000 UTC m=+19.646830737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:06.238254 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:06.237647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:06.238254 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.237806    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:06.238254 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.237829    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:06.238254 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.237842    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:06.238254 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.237900    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:14.2378818 +0000 UTC m=+19.848561765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:06.471888 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:06.471817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:06.471888 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:06.471862    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:06.472082 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.471939    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:06.472082 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.472016    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:06.472160 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:06.472105    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:06.472211 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:06.472193    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:08.471085 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:08.471048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:08.471598 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:08.471048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:08.471598 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:08.471197    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:08.471598 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:08.471055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:08.471598 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:08.471239    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:08.471598 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:08.471356    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:10.471188 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:10.471147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:10.471587 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:10.471146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:10.471587 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:10.471289    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:10.471587 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:10.471159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:10.471587 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:10.471355    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:10.471587 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:10.471383    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:12.470899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:12.470867    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:12.471362 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:12.470871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:12.471362 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:12.471003    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:12.471362 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:12.471161    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:12.471362 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:12.471140    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:12.471362 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:12.471214    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:14.095652 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:14.095615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:14.096147 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:14.095669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:14.096147 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.095747    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:14.096147 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.095826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:30.095805057 +0000 UTC m=+35.706485000 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:14.096147 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.095872    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:14.096147 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.095948    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:30.095920753 +0000 UTC m=+35.706600702 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:14.297098 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:14.297055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:14.297255 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.297208    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:14.297255 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.297225    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:14.297255 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.297238    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:14.297396 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.297300    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:30.297282049 +0000 UTC m=+35.907962006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:14.470904 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:14.470814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:14.470904 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:14.470830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:14.471149 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:14.470814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:14.471149 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.470931    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:14.471149 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.471088    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:14.471276 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:14.471209    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:16.471691 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:16.471647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:16.472122 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:16.471759    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:16.472122 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:16.471834    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:16.472122 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:16.471945    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:16.472122 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:16.471987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:16.472122 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:16.472062    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:17.623486 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.623448    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wrrj9" event={"ID":"b39f1022-4cf2-4513-a0fb-92216b5a67e3","Type":"ContainerStarted","Data":"892f6ba171a3f1dc687e8b6c0357431f1a23d8cf68bb81273aee78198938a9ec"}
Sep 19 05:41:17.624979 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.624957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qp5nd" event={"ID":"98470b56-3241-4618-9bf4-d6578d14c33d","Type":"ContainerStarted","Data":"93dad3056ece48359eb3596bced35e0c9e08597bff1f12f277d79af63a26f98f"}
Sep 19 05:41:17.626463 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.626443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"47e6577d0cbba2aeb381bdfc968a0464a826b1d5c556c6fc9aa2a9b2615ed549"}
Sep 19 05:41:17.627775 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.627758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mcq9" event={"ID":"cbffbbbd-ec33-42ab-b0e6-65fd2297af7b","Type":"ContainerStarted","Data":"fbc6d2b6b3fcb294da347f5720c07bac986134c1e0b330431a894789e3e258cb"}
Sep 19 05:41:17.629155 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.629136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x" event={"ID":"701d115e-2e55-4744-9999-6f5f09155082","Type":"ContainerStarted","Data":"cd24234809355cf137f07309c893d772ab7e09dba50d225046a064c89260fa03"}
Sep 19 05:41:17.630569 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.630549    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8gb7c" event={"ID":"e349d24e-28fc-4824-adad-292ac8dee6e9","Type":"ContainerStarted","Data":"002eadcfdbb89d50dd488656a6630a8549566d3c1c8c0b64b6ccba6a984dbe21"}
Sep 19 05:41:17.631997 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.631970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerStarted","Data":"19a5577d60be297a67268df58ea95680294265026f1546ed41853cc79d27d006"}
Sep 19 05:41:17.633281 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.633261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pcmxb" event={"ID":"da9d45fc-33f1-4fad-b8ba-35bedabba559","Type":"ContainerStarted","Data":"00b0bef13b2030d66ed8c0020241cb585b71ca51ac428fb072488cd494373c04"}
Sep 19 05:41:17.644335 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.644288    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.ec2.internal" podStartSLOduration=20.644261033 podStartE2EDuration="20.644261033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:01.605622427 +0000 UTC m=+7.216302393" watchObservedRunningTime="2026-09-19 05:41:17.644261033 +0000 UTC m=+23.254940994"
Sep 19 05:41:17.644978 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.644946    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wrrj9" podStartSLOduration=4.063130212 podStartE2EDuration="21.644938596s" totalImagesPullingTime="17.581808384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:17.644334713 +0000 UTC m=+23.255014677" watchObservedRunningTime="2026-09-19 05:41:17.644938596 +0000 UTC m=+23.255618560"
Sep 19 05:41:17.695517 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.695473    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2mcq9" podStartSLOduration=4.104625797 podStartE2EDuration="21.695457025s" totalImagesPullingTime="17.590831228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:17.66925589 +0000 UTC m=+23.279935858" watchObservedRunningTime="2026-09-19 05:41:17.695457025 +0000 UTC m=+23.306137051"
Sep 19 05:41:17.733797 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.733533    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pcmxb" podStartSLOduration=4.162457269 podStartE2EDuration="21.733514269s" totalImagesPullingTime="17.571057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:17.714248599 +0000 UTC m=+23.324928565" watchObservedRunningTime="2026-09-19 05:41:17.733514269 +0000 UTC m=+23.344194250"
Sep 19 05:41:17.734554 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.734516    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qp5nd" podStartSLOduration=3.8418815630000003 podStartE2EDuration="21.73449963s" totalImagesPullingTime="17.892618067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:17.733356496 +0000 UTC m=+23.344036459" watchObservedRunningTime="2026-09-19 05:41:17.73449963 +0000 UTC m=+23.345179597"
Sep 19 05:41:17.752304 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:17.752256    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8gb7c" podStartSLOduration=8.129521025 podStartE2EDuration="21.752240723s" totalImagesPullingTime="13.622719698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:17.751762626 +0000 UTC m=+23.362442590" watchObservedRunningTime="2026-09-19 05:41:17.752240723 +0000 UTC m=+23.362920686"
Sep 19 05:41:18.471365 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.471158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:18.471553 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.471217    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:18.471553 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:18.471447    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:18.471553 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:18.471534    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:18.471656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.471250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:18.471656 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:18.471620    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:18.636769 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.636737    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b530891-1fbd-4286-a117-e437388721eb" containerID="19a5577d60be297a67268df58ea95680294265026f1546ed41853cc79d27d006" exitCode=0
Sep 19 05:41:18.637528 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.636792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerDied","Data":"19a5577d60be297a67268df58ea95680294265026f1546ed41853cc79d27d006"}
Sep 19 05:41:18.638422 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.638385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hm7m" event={"ID":"95f848fd-2aef-41c3-92b7-c622d84c6c85","Type":"ContainerStarted","Data":"9753641c98528eef1c0c8b99e457018302dea972af8287907b1b12ae18a2c9fa"}
Sep 19 05:41:18.641308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641292    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:41:18.641655 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641638    2581 generic.go:356] "Generic (PLEG): container finished" podID="95225afd-e021-41c3-8fcb-b99d36b67473" containerID="c3d0ffa7d5129ef32faa57db1b628892ef91043e022326eb7e9519a47f5004df" exitCode=1
Sep 19 05:41:18.641740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"5eee5936b02a6c061691880eaf05f5af9dbe674c335dae43de555f88a6e2a888"}
Sep 19 05:41:18.641740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"c1cd28d38fde68fd35de6349d51e644046b9d4f97ffa93ed76068b75e767c5a9"}
Sep 19 05:41:18.641801 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"21410499dbcd3e492952542f0a014f11470af9281a2d66b81d2d7727673150a6"}
Sep 19 05:41:18.641833 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"3f692c1ffb035e46535c292f7b2e7af7b84e701f7a488e5f12a54868463afb9f"}
Sep 19 05:41:18.641833 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.641830    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerDied","Data":"c3d0ffa7d5129ef32faa57db1b628892ef91043e022326eb7e9519a47f5004df"}
Sep 19 05:41:18.677582 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.677521    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5hm7m" podStartSLOduration=5.09636047 podStartE2EDuration="22.677505731s" totalImagesPullingTime="17.581145261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:18.677401225 +0000 UTC m=+24.288081209" watchObservedRunningTime="2026-09-19 05:41:18.677505731 +0000 UTC m=+24.288185694"
Sep 19 05:41:18.772101 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:18.772078    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 05:41:19.451252 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:19.451106    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T05:41:18.772094939Z","UUID":"40ab9e59-d77f-4574-bb1d-d9d40e9229a7","Handler":null,"Name":"","Endpoint":""}
Sep 19 05:41:19.453295 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:19.453270    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 05:41:19.453463 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:19.453308    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 05:41:19.648462 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:19.648416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x" event={"ID":"701d115e-2e55-4744-9999-6f5f09155082","Type":"ContainerStarted","Data":"b648398f5fe10a3a75f6a2c3652836790c5e050537ea33c19a43372e18260146"}
Sep 19 05:41:20.101659 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.101632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:41:20.102267 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.102246    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:41:20.471364 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.471276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:20.471508 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.471392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:20.471508 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:20.471408    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:20.471508 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:20.471486    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:20.471626 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.471495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:20.471626 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:20.471592    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:20.653845 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.653815    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:41:20.654235 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.654205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"3c5ca056195af52700aa8782f4f346f88aae979315cb71c5336b2f110e017a4c"}
Sep 19 05:41:20.656081 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.656057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x" event={"ID":"701d115e-2e55-4744-9999-6f5f09155082","Type":"ContainerStarted","Data":"2e2b5df48337b95a219921081065ff77bbe141f7b1dbdf8bbb56144a160fde23"}
Sep 19 05:41:20.656418 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.656405    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:41:20.656854 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.656837    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8gb7c"
Sep 19 05:41:20.717723 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:20.717661    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m692x" podStartSLOduration=4.869362917 podStartE2EDuration="24.717645947s" totalImagesPullingTime="19.84828303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:20.67711268 +0000 UTC m=+26.287792659" watchObservedRunningTime="2026-09-19 05:41:20.717645947 +0000 UTC m=+26.328325920"
Sep 19 05:41:22.471797 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:22.471689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:22.471797 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:22.471715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:22.472224 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:22.471844    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:22.472322 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:22.472297    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:22.472363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:22.472351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:22.472438 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:22.472422    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:23.665706 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:23.665560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:41:23.666358 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:23.666101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"8ba260067c20d168ba334be053e78a02140ca1a42fa4ced0ca44a7562d2f5709"}
Sep 19 05:41:23.666411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:23.666386    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:41:23.666460 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:23.666450    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:41:23.666564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:23.666552    2581 scope.go:117] "RemoveContainer" containerID="c3d0ffa7d5129ef32faa57db1b628892ef91043e022326eb7e9519a47f5004df"
Sep 19 05:41:23.682320 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:23.682289    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:41:24.471447 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.471419    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:24.471447 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.471433    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:24.471638 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.471433    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:24.471638 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:24.471515    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:24.471638 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:24.471609    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:24.471730 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:24.471675    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:24.671411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.671347    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:41:24.671841 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.671738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" event={"ID":"95225afd-e021-41c3-8fcb-b99d36b67473","Type":"ContainerStarted","Data":"885fee7ca696b2ca7d06ab589896f8177dcc8a853a1add53484b51a68b27380e"}
Sep 19 05:41:24.671937 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.671912    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:41:24.673792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.673770    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b530891-1fbd-4286-a117-e437388721eb" containerID="461d0eb72797f9b72ff97f84053a6598eef1fa9d554261a1e88c421befac0e49" exitCode=0
Sep 19 05:41:24.673896 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.673801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerDied","Data":"461d0eb72797f9b72ff97f84053a6598eef1fa9d554261a1e88c421befac0e49"}
Sep 19 05:41:24.688736 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.688715    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:41:24.709962 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:24.709918    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r9592" podStartSLOduration=11.078805485 podStartE2EDuration="28.709904009s" totalImagesPullingTime="17.631098524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:24.70645474 +0000 UTC m=+30.317134714" watchObservedRunningTime="2026-09-19 05:41:24.709904009 +0000 UTC m=+30.320583972"
Sep 19 05:41:25.355127 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:25.354910    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hq78s"]
Sep 19 05:41:25.355294 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:25.355238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:25.355336 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:25.355324    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:25.358375 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:25.358350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52bg8"]
Sep 19 05:41:25.358544 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:25.358448    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:25.358544 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:25.358532    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:25.359157 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:25.359134    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cnxxg"]
Sep 19 05:41:25.359397 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:25.359252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:25.359397 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:25.359364    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:26.471608 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:26.471576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:26.472067 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:26.471673    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:26.680533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:26.680453    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b530891-1fbd-4286-a117-e437388721eb" containerID="e10f197ec0abff06a5276bcd2beb20f7f0dbeecaeae7c145bd6e009e591d056d" exitCode=0
Sep 19 05:41:26.680666 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:26.680539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerDied","Data":"e10f197ec0abff06a5276bcd2beb20f7f0dbeecaeae7c145bd6e009e591d056d"}
Sep 19 05:41:27.471775 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:27.471687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:27.471775 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:27.471739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:27.472299 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:27.471836    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:27.472299 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:27.471948    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:28.471312 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:28.471277    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:28.471484 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:28.471383    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:28.688359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:28.688328    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b530891-1fbd-4286-a117-e437388721eb" containerID="2d260446cd80c67b8dc5ffc5086fe555c7d5215e6ac71db2d72f3e36c9478e5e" exitCode=0
Sep 19 05:41:28.688740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:28.688371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerDied","Data":"2d260446cd80c67b8dc5ffc5086fe555c7d5215e6ac71db2d72f3e36c9478e5e"}
Sep 19 05:41:29.471075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:29.470863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:29.471441 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:29.470918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:29.471441 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:29.471186    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnxxg" podUID="6c42040f-8e4f-4d40-b199-58def6a92b99"
Sep 19 05:41:29.471441 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:29.471254    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq78s" podUID="78f960ee-50d2-40db-a972-81da4ef0d346"
Sep 19 05:41:30.114549 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:30.114504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:30.114974 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:30.114561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:30.114974 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.114665    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:30.114974 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.114699    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:30.114974 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.114751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs podName:6c42040f-8e4f-4d40-b199-58def6a92b99 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:02.114733671 +0000 UTC m=+67.725413618 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs") pod "network-metrics-daemon-cnxxg" (UID: "6c42040f-8e4f-4d40-b199-58def6a92b99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:30.114974 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.114767    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret podName:78f960ee-50d2-40db-a972-81da4ef0d346 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:02.114760583 +0000 UTC m=+67.725440528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret") pod "global-pull-secret-syncer-hq78s" (UID: "78f960ee-50d2-40db-a972-81da4ef0d346") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:30.316012 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:30.315977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:30.316192 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.316148    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:30.316192 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.316168    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:30.316192 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.316177    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fm9s8 for pod openshift-network-diagnostics/network-check-target-52bg8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:30.316341 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.316241    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8 podName:03a64404-88ed-4436-b000-10679862a70b nodeName:}" failed. No retries permitted until 2026-09-19 05:42:02.316224893 +0000 UTC m=+67.926904852 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fm9s8" (UniqueName: "kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8") pod "network-check-target-52bg8" (UID: "03a64404-88ed-4436-b000-10679862a70b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:30.471462 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:30.471383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:30.471623 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:30.471515    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52bg8" podUID="03a64404-88ed-4436-b000-10679862a70b"
Sep 19 05:41:31.470474 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.470449    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-202.ec2.internal" event="NodeReady"
Sep 19 05:41:31.470881 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.470590    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 05:41:31.470881 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.470639    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:41:31.470881 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.470860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:41:31.474740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.474719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 05:41:31.475087 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.474929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 05:41:31.475690 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.475668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjt75\""
Sep 19 05:41:31.552088 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.552055    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"]
Sep 19 05:41:31.553955 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.553937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"
Sep 19 05:41:31.557221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.556963    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.557221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.556963    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9bn69\""
Sep 19 05:41:31.557464 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.557359    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-64wfl"]
Sep 19 05:41:31.559186 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.559167    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8667f66c84-kv9d7"]
Sep 19 05:41:31.559285 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.559270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.559917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.559894    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.560979 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.560961    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-959d65cbd-84n4l"]
Sep 19 05:41:31.562425 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.562407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.562602 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.562580    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.563724 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.563706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.565091 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.564274    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-79scr"]
Sep 19 05:41:31.565091 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.564354    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 19 05:41:31.565091 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.564599    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qch2c\""
Sep 19 05:41:31.565091 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.564649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 19 05:41:31.565091 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.564884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.566014 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.565994    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-htq54"]
Sep 19 05:41:31.566210 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.566189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.567817 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.567797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.569537 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.569521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"]
Sep 19 05:41:31.573834 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.573808    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 19 05:41:31.574607 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.574318    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 19 05:41:31.579016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.575268    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.579016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.575514    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 19 05:41:31.579016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.576567    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 19 05:41:31.579016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.576783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j5wqj\""
Sep 19 05:41:31.579016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.577892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 05:41:31.580146 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.579872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 19 05:41:31.580666 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.580643    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sp8wq\""
Sep 19 05:41:31.581464 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.581444    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbwrd\""
Sep 19 05:41:31.581647 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.581631    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.581789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.581765    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.581915    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.582085    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.582576    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.582844    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.582997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q45g7\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.583018    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 05:41:31.583696 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.583173    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 05:41:31.584049 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.583781    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-64wfl"]
Sep 19 05:41:31.584705 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.584683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-959d65cbd-84n4l"]
Sep 19 05:41:31.585669 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.585650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-79scr"]
Sep 19 05:41:31.587429 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.587411    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-htq54"]
Sep 19 05:41:31.587584 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.586824    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 05:41:31.588487 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.588435    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8667f66c84-kv9d7"]
Sep 19 05:41:31.604136 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.604113    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"]
Sep 19 05:41:31.605947 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.605927    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.609254 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.608892    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.609254 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.609126    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.609254 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.609133    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 19 05:41:31.609254 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.609145    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5vpxd\""
Sep 19 05:41:31.609254 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.609156    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 19 05:41:31.618997 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.618978    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"]
Sep 19 05:41:31.624528 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624504    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbv4h\" (UniqueName: \"kubernetes.io/projected/f9ee5577-b709-488c-a1da-75108799e433-kube-api-access-mbv4h\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.624638 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xqbh\" (UniqueName: \"kubernetes.io/projected/ebde495e-633a-4484-b386-4d650b0011ca-kube-api-access-5xqbh\") pod \"ingress-canary-79scr\" (UID: \"ebde495e-633a-4484-b386-4d650b0011ca\") " pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.624638 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-trusted-ca\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.624638 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.624638 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-installation-pull-secrets\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4xqr\" (UniqueName: \"kubernetes.io/projected/1c599698-b28e-4617-bbb1-45c8cfae0673-kube-api-access-h4xqr\") pod \"volume-data-source-validator-5c4db8bdf9-bnp4j\" (UID: \"1c599698-b28e-4617-bbb1-45c8cfae0673\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9ee5577-b709-488c-a1da-75108799e433-trusted-ca\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d640860a-d9d8-4b07-a981-5a3e114ec318-ca-trust-extracted\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-certificates\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-bound-sa-token\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.624829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xt2j\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-kube-api-access-9xt2j\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.625141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-stats-auth\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.625141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebde495e-633a-4484-b386-4d650b0011ca-cert\") pod \"ingress-canary-79scr\" (UID: \"ebde495e-633a-4484-b386-4d650b0011ca\") " pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.625141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.624984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-default-certificate\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.625141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.625011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwk9\" (UniqueName: \"kubernetes.io/projected/a4c18157-81fa-45a4-b112-7beaed10c0ef-kube-api-access-gnwk9\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.625141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.625058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-image-registry-private-configuration\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.625141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.625103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9ee5577-b709-488c-a1da-75108799e433-serving-cert\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.625342 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.625155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.625342 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.625181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9ee5577-b709-488c-a1da-75108799e433-config\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.725668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9ee5577-b709-488c-a1da-75108799e433-trusted-ca\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.725668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d640860a-d9d8-4b07-a981-5a3e114ec318-ca-trust-extracted\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.725668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-certificates\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.725668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-bound-sa-token\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xt2j\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-kube-api-access-9xt2j\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-stats-auth\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlzfx\" (UniqueName: \"kubernetes.io/projected/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-kube-api-access-zlzfx\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebde495e-633a-4484-b386-4d650b0011ca-cert\") pod \"ingress-canary-79scr\" (UID: \"ebde495e-633a-4484-b386-4d650b0011ca\") " pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-default-certificate\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwk9\" (UniqueName: \"kubernetes.io/projected/a4c18157-81fa-45a4-b112-7beaed10c0ef-kube-api-access-gnwk9\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-image-registry-private-configuration\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9ee5577-b709-488c-a1da-75108799e433-serving-cert\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9ee5577-b709-488c-a1da-75108799e433-config\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbv4h\" (UniqueName: \"kubernetes.io/projected/f9ee5577-b709-488c-a1da-75108799e433-kube-api-access-mbv4h\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.725993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6336f700-eea4-4f3a-ba95-a96d1167afc2-config-volume\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.726006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xqbh\" (UniqueName: \"kubernetes.io/projected/ebde495e-633a-4484-b386-4d650b0011ca-kube-api-access-5xqbh\") pod \"ingress-canary-79scr\" (UID: \"ebde495e-633a-4484-b386-4d650b0011ca\") " pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.726677 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-trusted-ca\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726677 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5ggb\" (UniqueName: \"kubernetes.io/projected/6336f700-eea4-4f3a-ba95-a96d1167afc2-kube-api-access-n5ggb\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.726677 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d640860a-d9d8-4b07-a981-5a3e114ec318-ca-trust-extracted\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726677 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.726172    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:31.726677 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.726189    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-959d65cbd-84n4l: secret "image-registry-tls" not found
Sep 19 05:41:31.726677 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.726252    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls podName:d640860a-d9d8-4b07-a981-5a3e114ec318 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.226233683 +0000 UTC m=+37.836913626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls") pod "image-registry-959d65cbd-84n4l" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318") : secret "image-registry-tls" not found
Sep 19 05:41:31.726875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.726875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9ee5577-b709-488c-a1da-75108799e433-trusted-ca\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.726875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-installation-pull-secrets\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.726875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6336f700-eea4-4f3a-ba95-a96d1167afc2-tmp-dir\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.727071 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.727071 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-certificates\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.727071 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.727071 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.726931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4xqr\" (UniqueName: \"kubernetes.io/projected/1c599698-b28e-4617-bbb1-45c8cfae0673-kube-api-access-h4xqr\") pod \"volume-data-source-validator-5c4db8bdf9-bnp4j\" (UID: \"1c599698-b28e-4617-bbb1-45c8cfae0673\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"
Sep 19 05:41:31.727071 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.726958    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.226940936 +0000 UTC m=+37.837620897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : configmap references non-existent config key: service-ca.crt
Sep 19 05:41:31.727622 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.727596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9ee5577-b709-488c-a1da-75108799e433-config\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.727722 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.727706    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 05:41:31.727784 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.727764    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.227746521 +0000 UTC m=+37.838426472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : secret "router-metrics-certs-default" not found
Sep 19 05:41:31.728749 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.728714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-trusted-ca\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.730596 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.730576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9ee5577-b709-488c-a1da-75108799e433-serving-cert\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.739057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.730740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-installation-pull-secrets\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.739057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.730855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebde495e-633a-4484-b386-4d650b0011ca-cert\") pod \"ingress-canary-79scr\" (UID: \"ebde495e-633a-4484-b386-4d650b0011ca\") " pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.739057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.731574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-image-registry-private-configuration\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.739057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.735323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-stats-auth\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.739057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.735506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-default-certificate\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.740588 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.740066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-bound-sa-token\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.740737 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.740710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4xqr\" (UniqueName: \"kubernetes.io/projected/1c599698-b28e-4617-bbb1-45c8cfae0673-kube-api-access-h4xqr\") pod \"volume-data-source-validator-5c4db8bdf9-bnp4j\" (UID: \"1c599698-b28e-4617-bbb1-45c8cfae0673\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"
Sep 19 05:41:31.741524 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.741493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbv4h\" (UniqueName: \"kubernetes.io/projected/f9ee5577-b709-488c-a1da-75108799e433-kube-api-access-mbv4h\") pod \"console-operator-fd4bb46c-64wfl\" (UID: \"f9ee5577-b709-488c-a1da-75108799e433\") " pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.742333 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.742309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xqbh\" (UniqueName: \"kubernetes.io/projected/ebde495e-633a-4484-b386-4d650b0011ca-kube-api-access-5xqbh\") pod \"ingress-canary-79scr\" (UID: \"ebde495e-633a-4484-b386-4d650b0011ca\") " pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.743083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.743061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwk9\" (UniqueName: \"kubernetes.io/projected/a4c18157-81fa-45a4-b112-7beaed10c0ef-kube-api-access-gnwk9\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:31.743443 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.743414    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xt2j\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-kube-api-access-9xt2j\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:31.828487 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6336f700-eea4-4f3a-ba95-a96d1167afc2-config-volume\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.828487 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5ggb\" (UniqueName: \"kubernetes.io/projected/6336f700-eea4-4f3a-ba95-a96d1167afc2-kube-api-access-n5ggb\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.828699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6336f700-eea4-4f3a-ba95-a96d1167afc2-tmp-dir\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.828699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.828699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.828699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlzfx\" (UniqueName: \"kubernetes.io/projected/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-kube-api-access-zlzfx\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.828699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.828958 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.828743    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:31.828958 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:31.828811    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls podName:6336f700-eea4-4f3a-ba95-a96d1167afc2 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.328793258 +0000 UTC m=+37.939473220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls") pod "dns-default-htq54" (UID: "6336f700-eea4-4f3a-ba95-a96d1167afc2") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:31.828958 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.828875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6336f700-eea4-4f3a-ba95-a96d1167afc2-tmp-dir\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.829152 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.829126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6336f700-eea4-4f3a-ba95-a96d1167afc2-config-volume\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.829262 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.829236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.831056 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.831018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.854897 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.854869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5ggb\" (UniqueName: \"kubernetes.io/projected/6336f700-eea4-4f3a-ba95-a96d1167afc2-kube-api-access-n5ggb\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:31.855442 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.855417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlzfx\" (UniqueName: \"kubernetes.io/projected/f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3-kube-api-access-zlzfx\") pod \"kube-storage-version-migrator-operator-55889f7b47-svbh9\" (UID: \"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:31.868421 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.868368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"
Sep 19 05:41:31.878891 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.878869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:31.904968 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.904934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-79scr"
Sep 19 05:41:31.919167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:31.919136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"
Sep 19 05:41:32.232211 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.232171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:32.232414 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.232290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:32.232414 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.232326    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:32.232414 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.232347    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-959d65cbd-84n4l: secret "image-registry-tls" not found
Sep 19 05:41:32.232414 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.232411    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls podName:d640860a-d9d8-4b07-a981-5a3e114ec318 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:33.232391335 +0000 UTC m=+38.843071302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls") pod "image-registry-959d65cbd-84n4l" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318") : secret "image-registry-tls" not found
Sep 19 05:41:32.232599 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.232420    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 05:41:32.232599 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.232333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:32.232599 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.232437    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:33.232420874 +0000 UTC m=+38.843100824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : configmap references non-existent config key: service-ca.crt
Sep 19 05:41:32.232599 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.232464    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:33.232453566 +0000 UTC m=+38.843133508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : secret "router-metrics-certs-default" not found
Sep 19 05:41:32.333749 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.333717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:32.333931 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.333847    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:32.333931 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:32.333907    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls podName:6336f700-eea4-4f3a-ba95-a96d1167afc2 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:33.333889491 +0000 UTC m=+38.944569433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls") pod "dns-default-htq54" (UID: "6336f700-eea4-4f3a-ba95-a96d1167afc2") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:32.471232 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.471194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:41:32.474315 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.474078    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 05:41:32.474315 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.474109    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2b8g\""
Sep 19 05:41:32.474835 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:32.474813    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 05:41:33.241215 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.241181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:33.241406 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.241298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:33.241406 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.241329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:33.241406 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.241358    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:33.241406 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.241381    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-959d65cbd-84n4l: secret "image-registry-tls" not found
Sep 19 05:41:33.241582 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.241433    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 05:41:33.241582 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.241464    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls podName:d640860a-d9d8-4b07-a981-5a3e114ec318 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:35.241445335 +0000 UTC m=+40.852125296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls") pod "image-registry-959d65cbd-84n4l" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318") : secret "image-registry-tls" not found
Sep 19 05:41:33.241582 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.241486    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:35.241475727 +0000 UTC m=+40.852155683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : configmap references non-existent config key: service-ca.crt
Sep 19 05:41:33.241582 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.241501    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:35.241493367 +0000 UTC m=+40.852173323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : secret "router-metrics-certs-default" not found
Sep 19 05:41:33.342972 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.342932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:33.343812 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.343332    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:33.343812 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:33.343416    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls podName:6336f700-eea4-4f3a-ba95-a96d1167afc2 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:35.343393743 +0000 UTC m=+40.954073701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls") pod "dns-default-htq54" (UID: "6336f700-eea4-4f3a-ba95-a96d1167afc2") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:33.639258 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.639078    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-79scr"]
Sep 19 05:41:33.641550 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.641522    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9"]
Sep 19 05:41:33.642344 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.642324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j"]
Sep 19 05:41:33.643504 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.643485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-64wfl"]
Sep 19 05:41:33.657985 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:33.657958    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c599698_b28e_4617_bbb1_45c8cfae0673.slice/crio-e5950d0c5f1b04ffee67c2443ca508f0d2e9565b77fbd189b5ad7931f12b9ee1 WatchSource:0}: Error finding container e5950d0c5f1b04ffee67c2443ca508f0d2e9565b77fbd189b5ad7931f12b9ee1: Status 404 returned error can't find the container with id e5950d0c5f1b04ffee67c2443ca508f0d2e9565b77fbd189b5ad7931f12b9ee1
Sep 19 05:41:33.658293 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:33.658261    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebde495e_633a_4484_b386_4d650b0011ca.slice/crio-d195da028aeafb1b23459a4e0780449cc0cc7429f2b088dcc0c8e68df671c9f4 WatchSource:0}: Error finding container d195da028aeafb1b23459a4e0780449cc0cc7429f2b088dcc0c8e68df671c9f4: Status 404 returned error can't find the container with id d195da028aeafb1b23459a4e0780449cc0cc7429f2b088dcc0c8e68df671c9f4
Sep 19 05:41:33.658597 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:33.658578    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ee5577_b709_488c_a1da_75108799e433.slice/crio-b10d0a855a4146567d28b87950df7c2c38145495323d9ecd444f55a3812bdf47 WatchSource:0}: Error finding container b10d0a855a4146567d28b87950df7c2c38145495323d9ecd444f55a3812bdf47: Status 404 returned error can't find the container with id b10d0a855a4146567d28b87950df7c2c38145495323d9ecd444f55a3812bdf47
Sep 19 05:41:33.658866 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:33.658791    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e6ed25_f8e9_4d1e_ab7b_f5d18a2483e3.slice/crio-b002988de0eca341059d051805be92c48ca372119029ba2b12122cf7e8840387 WatchSource:0}: Error finding container b002988de0eca341059d051805be92c48ca372119029ba2b12122cf7e8840387: Status 404 returned error can't find the container with id b002988de0eca341059d051805be92c48ca372119029ba2b12122cf7e8840387
Sep 19 05:41:33.702970 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.702944    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b530891-1fbd-4286-a117-e437388721eb" containerID="a573bdff3f864dda29c7afcdd58ff610abb7906eadcb50bde951c95db0709f4b" exitCode=0
Sep 19 05:41:33.703103 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.703016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerDied","Data":"a573bdff3f864dda29c7afcdd58ff610abb7906eadcb50bde951c95db0709f4b"}
Sep 19 05:41:33.704157 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.704137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" event={"ID":"f9ee5577-b709-488c-a1da-75108799e433","Type":"ContainerStarted","Data":"b10d0a855a4146567d28b87950df7c2c38145495323d9ecd444f55a3812bdf47"}
Sep 19 05:41:33.705301 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.705270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9" event={"ID":"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3","Type":"ContainerStarted","Data":"b002988de0eca341059d051805be92c48ca372119029ba2b12122cf7e8840387"}
Sep 19 05:41:33.706422 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.706406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-79scr" event={"ID":"ebde495e-633a-4484-b386-4d650b0011ca","Type":"ContainerStarted","Data":"d195da028aeafb1b23459a4e0780449cc0cc7429f2b088dcc0c8e68df671c9f4"}
Sep 19 05:41:33.709532 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:33.708080    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j" event={"ID":"1c599698-b28e-4617-bbb1-45c8cfae0673","Type":"ContainerStarted","Data":"e5950d0c5f1b04ffee67c2443ca508f0d2e9565b77fbd189b5ad7931f12b9ee1"}
Sep 19 05:41:34.719359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:34.718644    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b530891-1fbd-4286-a117-e437388721eb" containerID="95a23af8ceea7e28d6b439f123bb9f7066274b21dc8674a24a6cccadf44d0e93" exitCode=0
Sep 19 05:41:34.719359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:34.718705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerDied","Data":"95a23af8ceea7e28d6b439f123bb9f7066274b21dc8674a24a6cccadf44d0e93"}
Sep 19 05:41:35.261690 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.261578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:35.261690 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.261641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:35.261926 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.261697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:35.261926 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.261831    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:35.261926 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.261847    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-959d65cbd-84n4l: secret "image-registry-tls" not found
Sep 19 05:41:35.261926 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.261905    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls podName:d640860a-d9d8-4b07-a981-5a3e114ec318 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:39.26188516 +0000 UTC m=+44.872565124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls") pod "image-registry-959d65cbd-84n4l" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318") : secret "image-registry-tls" not found
Sep 19 05:41:35.262381 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.262353    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 05:41:35.262505 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.262410    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:39.262394225 +0000 UTC m=+44.873074184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : secret "router-metrics-certs-default" not found
Sep 19 05:41:35.262505 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.262482    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:39.262472034 +0000 UTC m=+44.873151981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : configmap references non-existent config key: service-ca.crt
Sep 19 05:41:35.362247 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.362208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:35.362408 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.362357    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:35.362463 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:35.362419    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls podName:6336f700-eea4-4f3a-ba95-a96d1167afc2 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:39.362400203 +0000 UTC m=+44.973080158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls") pod "dns-default-htq54" (UID: "6336f700-eea4-4f3a-ba95-a96d1167afc2") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:35.728695 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.728659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" event={"ID":"2b530891-1fbd-4286-a117-e437388721eb","Type":"ContainerStarted","Data":"15198289176cfc9ad322af0798c1b50556b800075f84435b15e153dffe8598d5"}
Sep 19 05:41:35.730365 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.730338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j" event={"ID":"1c599698-b28e-4617-bbb1-45c8cfae0673","Type":"ContainerStarted","Data":"ad11da378c6b23d035cb7f7fd5c7ad1da72e81ecedcff559643897879e949832"}
Sep 19 05:41:35.752410 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.752359    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xb7qg" podStartSLOduration=10.542902423 podStartE2EDuration="39.752340308s" totalImagesPullingTime="29.209437885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.751130995 +0000 UTC m=+41.361810990" watchObservedRunningTime="2026-09-19 05:41:35.752340308 +0000 UTC m=+41.363020269"
Sep 19 05:41:35.766732 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:35.766689    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-bnp4j" podStartSLOduration=3.405780975 podStartE2EDuration="4.766674046s" totalImagesPullingTime="1.360893071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.766184477 +0000 UTC m=+41.376864462" watchObservedRunningTime="2026-09-19 05:41:35.766674046 +0000 UTC m=+41.377354010"
Sep 19 05:41:36.633242 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.633188    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p8lb8"]
Sep 19 05:41:36.639367 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.639335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.642794 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.642769    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 05:41:36.643107 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.643085    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 05:41:36.643223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.643106    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 05:41:36.643964 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.643947    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 05:41:36.644098 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.643953    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s598g\""
Sep 19 05:41:36.648413 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.648392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8lb8"]
Sep 19 05:41:36.774776 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.774720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.775191 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.774855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.775191 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.774903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53392f56-cf45-4b07-8cfc-fc8d56369ff9-crio-socket\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.775191 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.774945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d446s\" (UniqueName: \"kubernetes.io/projected/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-api-access-d446s\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.775191 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.775103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53392f56-cf45-4b07-8cfc-fc8d56369ff9-data-volume\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.876392 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.876356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.876578 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.876474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.876578 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.876513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53392f56-cf45-4b07-8cfc-fc8d56369ff9-crio-socket\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.876578 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.876555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d446s\" (UniqueName: \"kubernetes.io/projected/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-api-access-d446s\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.876741 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.876631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53392f56-cf45-4b07-8cfc-fc8d56369ff9-data-volume\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.876848 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:36.876821    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:36.876954 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:36.876894    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:37.376877593 +0000 UTC m=+42.987557553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:36.877049 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.876990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53392f56-cf45-4b07-8cfc-fc8d56369ff9-data-volume\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.877136 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:36.877119    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:36.877213 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:36.877186    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:37.377169122 +0000 UTC m=+42.987849079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:36.877324 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.877305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53392f56-cf45-4b07-8cfc-fc8d56369ff9-crio-socket\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:36.897198 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:36.897119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d446s\" (UniqueName: \"kubernetes.io/projected/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-api-access-d446s\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:37.380874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.380831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:37.381128 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.380930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:37.381128 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:37.380983    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:37.381128 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:37.380998    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:37.381128 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:37.381081    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:38.381057887 +0000 UTC m=+43.991737846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:37.381128 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:37.381107    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:38.381095492 +0000 UTC m=+43.991775453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:37.737132 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.737099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-79scr" event={"ID":"ebde495e-633a-4484-b386-4d650b0011ca","Type":"ContainerStarted","Data":"9c6dc8b29d578ce99f33225ac0e5812c6879d72c22362262fbc2958a2e1cc7b8"}
Sep 19 05:41:37.738782 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.738764    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/0.log"
Sep 19 05:41:37.738898 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.738797    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9ee5577-b709-488c-a1da-75108799e433" containerID="a40a35efa0446c0d7c1bb3bd0157dd6b3568c1b06f82dc5020cce054153b4e1f" exitCode=255
Sep 19 05:41:37.738898 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.738852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" event={"ID":"f9ee5577-b709-488c-a1da-75108799e433","Type":"ContainerDied","Data":"a40a35efa0446c0d7c1bb3bd0157dd6b3568c1b06f82dc5020cce054153b4e1f"}
Sep 19 05:41:37.739142 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.739128    2581 scope.go:117] "RemoveContainer" containerID="a40a35efa0446c0d7c1bb3bd0157dd6b3568c1b06f82dc5020cce054153b4e1f"
Sep 19 05:41:37.740435 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.740411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9" event={"ID":"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3","Type":"ContainerStarted","Data":"fbc6120e1007d9b023837875f13ed0ecd8a78c8cf0ee116abe63c0f5885f8e1f"}
Sep 19 05:41:37.770641 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.770595    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-79scr" podStartSLOduration=3.394024419 podStartE2EDuration="6.770581939s" totalImagesPullingTime="3.37655752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:37.770104701 +0000 UTC m=+43.380784686" watchObservedRunningTime="2026-09-19 05:41:37.770581939 +0000 UTC m=+43.381261893"
Sep 19 05:41:37.796018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:37.795971    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9" podStartSLOduration=3.4239273900000002 podStartE2EDuration="6.795952749s" totalImagesPullingTime="3.372025359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:37.794395924 +0000 UTC m=+43.405075906" watchObservedRunningTime="2026-09-19 05:41:37.795952749 +0000 UTC m=+43.406632713"
Sep 19 05:41:38.326766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.326741    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pcmxb_da9d45fc-33f1-4fad-b8ba-35bedabba559/dns-node-resolver/0.log"
Sep 19 05:41:38.389001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.388968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:38.389167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.389028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:38.389167 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:38.389109    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:38.389167 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:38.389163    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:38.389265 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:38.389181    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:40.389161354 +0000 UTC m=+45.999841296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:38.389265 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:38.389204    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:40.389192779 +0000 UTC m=+45.999872724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:38.745596 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.745522    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:41:38.745955 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.745941    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/0.log"
Sep 19 05:41:38.746011 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.745978    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9ee5577-b709-488c-a1da-75108799e433" containerID="d5cfea8bcf90f0943d9fe041918e7aa6969913fa572a0c886d5415ec0d30622d" exitCode=255
Sep 19 05:41:38.746109 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.746087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" event={"ID":"f9ee5577-b709-488c-a1da-75108799e433","Type":"ContainerDied","Data":"d5cfea8bcf90f0943d9fe041918e7aa6969913fa572a0c886d5415ec0d30622d"}
Sep 19 05:41:38.746179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.746142    2581 scope.go:117] "RemoveContainer" containerID="a40a35efa0446c0d7c1bb3bd0157dd6b3568c1b06f82dc5020cce054153b4e1f"
Sep 19 05:41:38.746371 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:38.746352    2581 scope.go:117] "RemoveContainer" containerID="d5cfea8bcf90f0943d9fe041918e7aa6969913fa572a0c886d5415ec0d30622d"
Sep 19 05:41:38.746574 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:38.746552    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-64wfl_openshift-console-operator(f9ee5577-b709-488c-a1da-75108799e433)\"" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" podUID="f9ee5577-b709-488c-a1da-75108799e433"
Sep 19 05:41:39.126561 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.126535    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wrrj9_b39f1022-4cf2-4513-a0fb-92216b5a67e3/node-ca/0.log"
Sep 19 05:41:39.295453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.295416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:39.295453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.295459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.295482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.295587    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.295617    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.295629    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-959d65cbd-84n4l: secret "image-registry-tls" not found
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.295597    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.295578311 +0000 UTC m=+52.906258257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : configmap references non-existent config key: service-ca.crt
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.295662    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs podName:a4c18157-81fa-45a4-b112-7beaed10c0ef nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.295640658 +0000 UTC m=+52.906320599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs") pod "router-default-8667f66c84-kv9d7" (UID: "a4c18157-81fa-45a4-b112-7beaed10c0ef") : secret "router-metrics-certs-default" not found
Sep 19 05:41:39.295705 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.295681    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls podName:d640860a-d9d8-4b07-a981-5a3e114ec318 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.295671457 +0000 UTC m=+52.906351404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls") pod "image-registry-959d65cbd-84n4l" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318") : secret "image-registry-tls" not found
Sep 19 05:41:39.396571 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.396484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:39.396707 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.396641    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:39.396745 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.396711    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls podName:6336f700-eea4-4f3a-ba95-a96d1167afc2 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.396694499 +0000 UTC m=+53.007374441 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls") pod "dns-default-htq54" (UID: "6336f700-eea4-4f3a-ba95-a96d1167afc2") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:39.731499 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.731429    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-79scr_ebde495e-633a-4484-b386-4d650b0011ca/serve-healthcheck-canary/0.log"
Sep 19 05:41:39.749903 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.749875    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:41:39.750229 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:39.750214    2581 scope.go:117] "RemoveContainer" containerID="d5cfea8bcf90f0943d9fe041918e7aa6969913fa572a0c886d5415ec0d30622d"
Sep 19 05:41:39.750387 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:39.750370    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-64wfl_openshift-console-operator(f9ee5577-b709-488c-a1da-75108799e433)\"" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" podUID="f9ee5577-b709-488c-a1da-75108799e433"
Sep 19 05:41:40.405749 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:40.405710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:40.406163 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:40.405783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:40.406163 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:40.405838    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:40.406163 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:40.405900    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:40.406163 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:40.405907    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:44.405892174 +0000 UTC m=+50.016572134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:40.406163 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:40.405939    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:44.405928967 +0000 UTC m=+50.016608910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:41.879420 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:41.879381    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:41.879827 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:41.879438    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:41.879827 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:41.879773    2581 scope.go:117] "RemoveContainer" containerID="d5cfea8bcf90f0943d9fe041918e7aa6969913fa572a0c886d5415ec0d30622d"
Sep 19 05:41:41.879960 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:41.879938    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-64wfl_openshift-console-operator(f9ee5577-b709-488c-a1da-75108799e433)\"" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" podUID="f9ee5577-b709-488c-a1da-75108799e433"
Sep 19 05:41:44.439443 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:44.439398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:44.439843 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:44.439540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:44.439843 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:44.439566    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:44.439843 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:44.439622    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:44.439843 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:44.439659    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:52.439639379 +0000 UTC m=+58.050319335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:44.439843 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:41:44.439676    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm podName:53392f56-cf45-4b07-8cfc-fc8d56369ff9 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:52.439668972 +0000 UTC m=+58.050348914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p8lb8" (UID: "53392f56-cf45-4b07-8cfc-fc8d56369ff9") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:47.365669 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.365634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:47.365669 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.365674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:47.366115 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.365695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:47.366379 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.366360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4c18157-81fa-45a4-b112-7beaed10c0ef-service-ca-bundle\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:47.368275 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.368240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c18157-81fa-45a4-b112-7beaed10c0ef-metrics-certs\") pod \"router-default-8667f66c84-kv9d7\" (UID: \"a4c18157-81fa-45a4-b112-7beaed10c0ef\") " pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:47.368386 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.368309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"image-registry-959d65cbd-84n4l\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") " pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:47.466246 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.466203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:47.468777 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.468747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6336f700-eea4-4f3a-ba95-a96d1167afc2-metrics-tls\") pod \"dns-default-htq54\" (UID: \"6336f700-eea4-4f3a-ba95-a96d1167afc2\") " pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:47.491009 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.490977    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:47.498147 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.498128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:47.512802 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.512777    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:47.805333 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.805299    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8667f66c84-kv9d7"]
Sep 19 05:41:47.809049 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:47.809004    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c18157_81fa_45a4_b112_7beaed10c0ef.slice/crio-90fa17fe093b395175ed1632c2cc7e89d310cac4784e8d2d74444b75d05d7729 WatchSource:0}: Error finding container 90fa17fe093b395175ed1632c2cc7e89d310cac4784e8d2d74444b75d05d7729: Status 404 returned error can't find the container with id 90fa17fe093b395175ed1632c2cc7e89d310cac4784e8d2d74444b75d05d7729
Sep 19 05:41:47.830950 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.830922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-959d65cbd-84n4l"]
Sep 19 05:41:47.833795 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:47.833772    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd640860a_d9d8_4b07_a981_5a3e114ec318.slice/crio-d56a73e8fe5cfc235043e482249655bd7e5d502933c2e3a3770f1eac213bc193 WatchSource:0}: Error finding container d56a73e8fe5cfc235043e482249655bd7e5d502933c2e3a3770f1eac213bc193: Status 404 returned error can't find the container with id d56a73e8fe5cfc235043e482249655bd7e5d502933c2e3a3770f1eac213bc193
Sep 19 05:41:47.861641 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:47.861604    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-htq54"]
Sep 19 05:41:47.864176 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:47.864153    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6336f700_eea4_4f3a_ba95_a96d1167afc2.slice/crio-5e544efd9593629dbbb0a66ec9f29628d5766334d032e379a3757ef4de7f8bd5 WatchSource:0}: Error finding container 5e544efd9593629dbbb0a66ec9f29628d5766334d032e379a3757ef4de7f8bd5: Status 404 returned error can't find the container with id 5e544efd9593629dbbb0a66ec9f29628d5766334d032e379a3757ef4de7f8bd5
Sep 19 05:41:48.776113 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.776073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-959d65cbd-84n4l" event={"ID":"d640860a-d9d8-4b07-a981-5a3e114ec318","Type":"ContainerStarted","Data":"b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380"}
Sep 19 05:41:48.776590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.776145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-959d65cbd-84n4l" event={"ID":"d640860a-d9d8-4b07-a981-5a3e114ec318","Type":"ContainerStarted","Data":"d56a73e8fe5cfc235043e482249655bd7e5d502933c2e3a3770f1eac213bc193"}
Sep 19 05:41:48.776590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.776171    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:41:48.777348 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.777314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-htq54" event={"ID":"6336f700-eea4-4f3a-ba95-a96d1167afc2","Type":"ContainerStarted","Data":"5e544efd9593629dbbb0a66ec9f29628d5766334d032e379a3757ef4de7f8bd5"}
Sep 19 05:41:48.778974 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.778949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8667f66c84-kv9d7" event={"ID":"a4c18157-81fa-45a4-b112-7beaed10c0ef","Type":"ContainerStarted","Data":"c7823b1487a787c02fe6b165f6e42c88dc9eb594b82ba6948902566d5ef4b776"}
Sep 19 05:41:48.779117 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.778991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8667f66c84-kv9d7" event={"ID":"a4c18157-81fa-45a4-b112-7beaed10c0ef","Type":"ContainerStarted","Data":"90fa17fe093b395175ed1632c2cc7e89d310cac4784e8d2d74444b75d05d7729"}
Sep 19 05:41:48.802983 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.802919    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-959d65cbd-84n4l" podStartSLOduration=17.802902448 podStartE2EDuration="17.802902448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:48.802262967 +0000 UTC m=+54.412942950" watchObservedRunningTime="2026-09-19 05:41:48.802902448 +0000 UTC m=+54.413582412"
Sep 19 05:41:48.829585 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:48.829532    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-8667f66c84-kv9d7" podStartSLOduration=17.829514466 podStartE2EDuration="17.829514466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:48.827454457 +0000 UTC m=+54.438134441" watchObservedRunningTime="2026-09-19 05:41:48.829514466 +0000 UTC m=+54.440194430"
Sep 19 05:41:49.492221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:49.492077    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:49.495073 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:49.495026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:49.782735 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:49.782707    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:49.783799 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:49.783778    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8667f66c84-kv9d7"
Sep 19 05:41:50.787655 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:50.787610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-htq54" event={"ID":"6336f700-eea4-4f3a-ba95-a96d1167afc2","Type":"ContainerStarted","Data":"2f959dd936d59bec91944a5772c09e59a36c02e504b2d7e3d97012632a1447c2"}
Sep 19 05:41:50.788147 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:50.787671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-htq54" event={"ID":"6336f700-eea4-4f3a-ba95-a96d1167afc2","Type":"ContainerStarted","Data":"e1385bb24cad45c7f0da7b7c26435e34c9fe6e1307f4afafd0fe91b201129aeb"}
Sep 19 05:41:50.788147 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:50.787827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-htq54"
Sep 19 05:41:50.807550 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:50.807497    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-htq54" podStartSLOduration=17.693659959 podStartE2EDuration="19.807481784s" totalImagesPullingTime="2.113821825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:50.80618932 +0000 UTC m=+56.416869304" watchObservedRunningTime="2026-09-19 05:41:50.807481784 +0000 UTC m=+56.418161748"
Sep 19 05:41:52.471236 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.471204    2581 scope.go:117] "RemoveContainer" containerID="d5cfea8bcf90f0943d9fe041918e7aa6969913fa572a0c886d5415ec0d30622d"
Sep 19 05:41:52.502647 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.502615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:52.502742 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.502698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:52.503274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.503241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53392f56-cf45-4b07-8cfc-fc8d56369ff9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:52.505125 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.505107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53392f56-cf45-4b07-8cfc-fc8d56369ff9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8lb8\" (UID: \"53392f56-cf45-4b07-8cfc-fc8d56369ff9\") " pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:52.550864 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.550833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8lb8"
Sep 19 05:41:52.797983 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.797959    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:41:52.798158 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.798014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" event={"ID":"f9ee5577-b709-488c-a1da-75108799e433","Type":"ContainerStarted","Data":"ece57389af47d3ee9e35ec8a82b4bc29baabf745afc7b0d7b50e0a58f0bfc6cc"}
Sep 19 05:41:52.798343 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.798317    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:52.818214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.818167    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl" podStartSLOduration=18.450963259 podStartE2EDuration="21.818153839s" totalImagesPullingTime="3.36719058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:52.81697588 +0000 UTC m=+58.427655856" watchObservedRunningTime="2026-09-19 05:41:52.818153839 +0000 UTC m=+58.428833823"
Sep 19 05:41:52.850466 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:52.850370    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8lb8"]
Sep 19 05:41:52.853696 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:41:52.853664    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53392f56_cf45_4b07_8cfc_fc8d56369ff9.slice/crio-2981396e8292382e1c3237f67fef858edad098e412766e7ee1afee099678313b WatchSource:0}: Error finding container 2981396e8292382e1c3237f67fef858edad098e412766e7ee1afee099678313b: Status 404 returned error can't find the container with id 2981396e8292382e1c3237f67fef858edad098e412766e7ee1afee099678313b
Sep 19 05:41:53.590176 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:53.590145    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-64wfl"
Sep 19 05:41:53.803233 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:53.803203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8lb8" event={"ID":"53392f56-cf45-4b07-8cfc-fc8d56369ff9","Type":"ContainerStarted","Data":"7558b92d2c2d89a0b28debe91edb1ebffea851fa217b3b1b71324f5392781e20"}
Sep 19 05:41:53.803382 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:53.803258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8lb8" event={"ID":"53392f56-cf45-4b07-8cfc-fc8d56369ff9","Type":"ContainerStarted","Data":"255a99e04fbaf07b36e25cb7e8f244d62344bf8df31d396df086b9120d7549ec"}
Sep 19 05:41:53.803382 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:53.803279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8lb8" event={"ID":"53392f56-cf45-4b07-8cfc-fc8d56369ff9","Type":"ContainerStarted","Data":"2981396e8292382e1c3237f67fef858edad098e412766e7ee1afee099678313b"}
Sep 19 05:41:55.812937 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:55.812899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8lb8" event={"ID":"53392f56-cf45-4b07-8cfc-fc8d56369ff9","Type":"ContainerStarted","Data":"9839d83c6026740d8dbf6a00426055a61d61243fd1788e1981bc258ac63d2eaf"}
Sep 19 05:41:55.841006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:55.840927    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p8lb8" podStartSLOduration=17.899432028 podStartE2EDuration="19.840911901s" totalImagesPullingTime="1.941479873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:55.839593041 +0000 UTC m=+61.450273039" watchObservedRunningTime="2026-09-19 05:41:55.840911901 +0000 UTC m=+61.451591864"
Sep 19 05:41:56.691759 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:41:56.691730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9592"
Sep 19 05:42:00.794493 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:00.794461    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-htq54"
Sep 19 05:42:01.396068 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.396018    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"]
Sep 19 05:42:01.398805 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.398784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.403056 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.403015    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 19 05:42:01.403187 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.403065    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 05:42:01.403187 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.403020    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 05:42:01.403187 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.403026    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 05:42:01.412164 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.412143    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"]
Sep 19 05:42:01.414290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.414272    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"]
Sep 19 05:42:01.414384 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.414363    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.416845 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:01.416813    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" type="*v1.Secret"
Sep 19 05:42:01.416980 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:01.416813    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-service-proxy-server-certificates\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" type="*v1.Secret"
Sep 19 05:42:01.416980 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:01.416814    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-ca\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" type="*v1.Secret"
Sep 19 05:42:01.416980 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:01.416923    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" type="*v1.Secret"
Sep 19 05:42:01.443727 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.443699    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"]
Sep 19 05:42:01.470682 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.470647    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-959d65cbd-84n4l"]
Sep 19 05:42:01.477757 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.477727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3f381e-2699-4dbd-8e66-975e781514f3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.477938 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.477768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/05097b82-4166-466d-a5a7-d8cc8f600ecd-klusterlet-config\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.477938 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.477829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.477938 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.477896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kjbf\" (UniqueName: \"kubernetes.io/projected/05097b82-4166-466d-a5a7-d8cc8f600ecd-kube-api-access-8kjbf\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.478161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.477955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.478161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.477977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.478161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.478057    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhtg7\" (UniqueName: \"kubernetes.io/projected/eb3f381e-2699-4dbd-8e66-975e781514f3-kube-api-access-jhtg7\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.478161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.478109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.478161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.478147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05097b82-4166-466d-a5a7-d8cc8f600ecd-tmp\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.578969 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.578937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.579203 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kjbf\" (UniqueName: \"kubernetes.io/projected/05097b82-4166-466d-a5a7-d8cc8f600ecd-kube-api-access-8kjbf\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.579285 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.579285 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.579389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhtg7\" (UniqueName: \"kubernetes.io/projected/eb3f381e-2699-4dbd-8e66-975e781514f3-kube-api-access-jhtg7\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.579389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.579491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05097b82-4166-466d-a5a7-d8cc8f600ecd-tmp\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.579491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3f381e-2699-4dbd-8e66-975e781514f3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.579491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/05097b82-4166-466d-a5a7-d8cc8f600ecd-klusterlet-config\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.579770 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.579749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05097b82-4166-466d-a5a7-d8cc8f600ecd-tmp\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.580089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.580066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3f381e-2699-4dbd-8e66-975e781514f3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.582054 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.582023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/05097b82-4166-466d-a5a7-d8cc8f600ecd-klusterlet-config\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.591818 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.591797    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79657c785d-ljnjl"]
Sep 19 05:42:01.593698 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.593684    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.597817 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.597794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kjbf\" (UniqueName: \"kubernetes.io/projected/05097b82-4166-466d-a5a7-d8cc8f600ecd-kube-api-access-8kjbf\") pod \"klusterlet-addon-workmgr-c5657d594-jl9wz\" (UID: \"05097b82-4166-466d-a5a7-d8cc8f600ecd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.598544 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.598525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhtg7\" (UniqueName: \"kubernetes.io/projected/eb3f381e-2699-4dbd-8e66-975e781514f3-kube-api-access-jhtg7\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:01.618270 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.618244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79657c785d-ljnjl"]
Sep 19 05:42:01.680855 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.680764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-registry-tls\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.680855 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.680850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2e80818-0d54-48a3-ba42-52d569f51fee-installation-pull-secrets\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.681067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.680888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-bound-sa-token\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.681067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.680922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2e80818-0d54-48a3-ba42-52d569f51fee-image-registry-private-configuration\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.681067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.681014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2e80818-0d54-48a3-ba42-52d569f51fee-ca-trust-extracted\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.681190 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.681069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2e80818-0d54-48a3-ba42-52d569f51fee-trusted-ca\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.681190 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.681094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2e80818-0d54-48a3-ba42-52d569f51fee-registry-certificates\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.681190 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.681119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc6jx\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-kube-api-access-zc6jx\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.707989 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.707967    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:01.782011 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.781971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2e80818-0d54-48a3-ba42-52d569f51fee-image-registry-private-configuration\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2e80818-0d54-48a3-ba42-52d569f51fee-ca-trust-extracted\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2e80818-0d54-48a3-ba42-52d569f51fee-trusted-ca\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2e80818-0d54-48a3-ba42-52d569f51fee-registry-certificates\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc6jx\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-kube-api-access-zc6jx\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-registry-tls\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2e80818-0d54-48a3-ba42-52d569f51fee-installation-pull-secrets\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782458 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-bound-sa-token\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.782793 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.782760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2e80818-0d54-48a3-ba42-52d569f51fee-ca-trust-extracted\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.783424 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.783401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2e80818-0d54-48a3-ba42-52d569f51fee-registry-certificates\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.783711 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.783690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2e80818-0d54-48a3-ba42-52d569f51fee-trusted-ca\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.784722 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.784701    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2e80818-0d54-48a3-ba42-52d569f51fee-image-registry-private-configuration\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.784855 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.784765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2e80818-0d54-48a3-ba42-52d569f51fee-installation-pull-secrets\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.784962 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.784943    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-registry-tls\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.790346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.790326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-bound-sa-token\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.790455 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.790418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc6jx\" (UniqueName: \"kubernetes.io/projected/e2e80818-0d54-48a3-ba42-52d569f51fee-kube-api-access-zc6jx\") pod \"image-registry-79657c785d-ljnjl\" (UID: \"e2e80818-0d54-48a3-ba42-52d569f51fee\") " pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.911017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.910989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:01.980707 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:01.980341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"]
Sep 19 05:42:02.185089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.185064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79657c785d-ljnjl"]
Sep 19 05:42:02.185382 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.185364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:42:02.185417 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.185407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:42:02.187384 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:42:02.187358    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e80818_0d54_48a3_ba42_52d569f51fee.slice/crio-56de2095219b00cbeb751e00c048229f410a87b88fb19c5c8eb625476b476a05 WatchSource:0}: Error finding container 56de2095219b00cbeb751e00c048229f410a87b88fb19c5c8eb625476b476a05: Status 404 returned error can't find the container with id 56de2095219b00cbeb751e00c048229f410a87b88fb19c5c8eb625476b476a05
Sep 19 05:42:02.188129 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.188101    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 05:42:02.188129 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.188124    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 05:42:02.197937 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.197918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c42040f-8e4f-4d40-b199-58def6a92b99-metrics-certs\") pod \"network-metrics-daemon-cnxxg\" (UID: \"6c42040f-8e4f-4d40-b199-58def6a92b99\") " pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:42:02.198010 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.197920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78f960ee-50d2-40db-a972-81da4ef0d346-original-pull-secret\") pod \"global-pull-secret-syncer-hq78s\" (UID: \"78f960ee-50d2-40db-a972-81da4ef0d346\") " pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:42:02.386180 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.386145    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjt75\""
Sep 19 05:42:02.386336 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.386320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:42:02.388607 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.388589    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 05:42:02.391452 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.391424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq78s"
Sep 19 05:42:02.394188 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.394162    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnxxg"
Sep 19 05:42:02.400285 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.400265    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 05:42:02.410758 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.410731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9s8\" (UniqueName: \"kubernetes.io/projected/03a64404-88ed-4436-b000-10679862a70b-kube-api-access-fm9s8\") pod \"network-check-target-52bg8\" (UID: \"03a64404-88ed-4436-b000-10679862a70b\") " pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:42:02.485765 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.485731    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2b8g\""
Sep 19 05:42:02.493728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.493693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:42:02.579949 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.579898    2581 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-ca: failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.579949 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.579941    2581 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert: failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.580203 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.580002    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub podName:eb3f381e-2699-4dbd-8e66-975e781514f3 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:03.079982478 +0000 UTC m=+68.690662423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "hub" (UniqueName: "kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub") pod "cluster-proxy-proxy-agent-855cf64699-sgwc4" (UID: "eb3f381e-2699-4dbd-8e66-975e781514f3") : failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.580203 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.579916    2581 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-service-proxy-server-certificates: failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.580203 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.580016    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-ca podName:eb3f381e-2699-4dbd-8e66-975e781514f3 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:03.080010087 +0000 UTC m=+68.690690029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ca" (UniqueName: "kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-ca") pod "cluster-proxy-proxy-agent-855cf64699-sgwc4" (UID: "eb3f381e-2699-4dbd-8e66-975e781514f3") : failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.580203 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.579916    2581 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-hub-kubeconfig: failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.580203 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.580065    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub-kubeconfig podName:eb3f381e-2699-4dbd-8e66-975e781514f3 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:03.080059216 +0000 UTC m=+68.690739158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "hub-kubeconfig" (UniqueName: "kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub-kubeconfig") pod "cluster-proxy-proxy-agent-855cf64699-sgwc4" (UID: "eb3f381e-2699-4dbd-8e66-975e781514f3") : failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.580203 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:02.580077    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-service-proxy-server-cert podName:eb3f381e-2699-4dbd-8e66-975e781514f3 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:03.080069826 +0000 UTC m=+68.690749768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-proxy-server-cert" (UniqueName: "kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-service-proxy-server-cert") pod "cluster-proxy-proxy-agent-855cf64699-sgwc4" (UID: "eb3f381e-2699-4dbd-8e66-975e781514f3") : failed to sync secret cache: timed out waiting for the condition
Sep 19 05:42:02.620875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.620846    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 19 05:42:02.679444 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.679419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hq78s"]
Sep 19 05:42:02.681559 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:42:02.681529    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f960ee_50d2_40db_a972_81da4ef0d346.slice/crio-7a057a38fde7184e844973b0b63815ad6058a14c6a886db1452229969ae058e3 WatchSource:0}: Error finding container 7a057a38fde7184e844973b0b63815ad6058a14c6a886db1452229969ae058e3: Status 404 returned error can't find the container with id 7a057a38fde7184e844973b0b63815ad6058a14c6a886db1452229969ae058e3
Sep 19 05:42:02.684336 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.683922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cnxxg"]
Sep 19 05:42:02.687168 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:42:02.687144    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c42040f_8e4f_4d40_b199_58def6a92b99.slice/crio-6290e8587749b5132fe536cbaa25d0f3c4fd6a5ba2c8cb69b7b2d5e1a46e2266 WatchSource:0}: Error finding container 6290e8587749b5132fe536cbaa25d0f3c4fd6a5ba2c8cb69b7b2d5e1a46e2266: Status 404 returned error can't find the container with id 6290e8587749b5132fe536cbaa25d0f3c4fd6a5ba2c8cb69b7b2d5e1a46e2266
Sep 19 05:42:02.764255 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.764166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52bg8"]
Sep 19 05:42:02.767070 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:42:02.767023    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a64404_88ed_4436_b000_10679862a70b.slice/crio-77b4aa42d2ec3306854949fba1ea5dfcfe1fe5df7914ee44c4ff0d49fbb3b994 WatchSource:0}: Error finding container 77b4aa42d2ec3306854949fba1ea5dfcfe1fe5df7914ee44c4ff0d49fbb3b994: Status 404 returned error can't find the container with id 77b4aa42d2ec3306854949fba1ea5dfcfe1fe5df7914ee44c4ff0d49fbb3b994
Sep 19 05:42:02.841304 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.838446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52bg8" event={"ID":"03a64404-88ed-4436-b000-10679862a70b","Type":"ContainerStarted","Data":"77b4aa42d2ec3306854949fba1ea5dfcfe1fe5df7914ee44c4ff0d49fbb3b994"}
Sep 19 05:42:02.841485 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.841384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnxxg" event={"ID":"6c42040f-8e4f-4d40-b199-58def6a92b99","Type":"ContainerStarted","Data":"6290e8587749b5132fe536cbaa25d0f3c4fd6a5ba2c8cb69b7b2d5e1a46e2266"}
Sep 19 05:42:02.842988 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.842960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz" event={"ID":"05097b82-4166-466d-a5a7-d8cc8f600ecd","Type":"ContainerStarted","Data":"b6bc9d28405c71645ec26d56df29c7482ec53cd44b2e91db0e38c832849a6c1a"}
Sep 19 05:42:02.844283 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.844252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hq78s" event={"ID":"78f960ee-50d2-40db-a972-81da4ef0d346","Type":"ContainerStarted","Data":"7a057a38fde7184e844973b0b63815ad6058a14c6a886db1452229969ae058e3"}
Sep 19 05:42:02.845962 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.845934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79657c785d-ljnjl" event={"ID":"e2e80818-0d54-48a3-ba42-52d569f51fee","Type":"ContainerStarted","Data":"d8fa99edbd372664a335e4c1c5ae7df5e438813b9999be0e76c67882e5a9174a"}
Sep 19 05:42:02.846095 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.845982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79657c785d-ljnjl" event={"ID":"e2e80818-0d54-48a3-ba42-52d569f51fee","Type":"ContainerStarted","Data":"56de2095219b00cbeb751e00c048229f410a87b88fb19c5c8eb625476b476a05"}
Sep 19 05:42:02.846728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.846712    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:02.870396 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.870341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79657c785d-ljnjl" podStartSLOduration=1.8703228680000001 podStartE2EDuration="1.870322868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:02.869787573 +0000 UTC m=+68.480467572" watchObservedRunningTime="2026-09-19 05:42:02.870322868 +0000 UTC m=+68.481002852"
Sep 19 05:42:02.905662 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.905626    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 19 05:42:02.962426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.962389    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 19 05:42:02.967318 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:02.967296    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 19 05:42:03.092983 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.092943    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.093189 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.092995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.093189 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.093064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.093189 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.093133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.096393 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.096365    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-ca\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.096700 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.096678    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.097252 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.097211    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.097363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.097259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/eb3f381e-2699-4dbd-8e66-975e781514f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855cf64699-sgwc4\" (UID: \"eb3f381e-2699-4dbd-8e66-975e781514f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.232994 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.232695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"
Sep 19 05:42:03.537796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.537760    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4"]
Sep 19 05:42:03.544296 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:42:03.544255    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb3f381e_2699_4dbd_8e66_975e781514f3.slice/crio-d6856261411b1250d0623ddd475c23c812f63c6c8d469e4e8cc282779c890527 WatchSource:0}: Error finding container d6856261411b1250d0623ddd475c23c812f63c6c8d469e4e8cc282779c890527: Status 404 returned error can't find the container with id d6856261411b1250d0623ddd475c23c812f63c6c8d469e4e8cc282779c890527
Sep 19 05:42:03.856535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:03.856433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4" event={"ID":"eb3f381e-2699-4dbd-8e66-975e781514f3","Type":"ContainerStarted","Data":"d6856261411b1250d0623ddd475c23c812f63c6c8d469e4e8cc282779c890527"}
Sep 19 05:42:05.872912 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:05.872873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnxxg" event={"ID":"6c42040f-8e4f-4d40-b199-58def6a92b99","Type":"ContainerStarted","Data":"7904499448a10be5fddff813783b0a168c7820491fcd7f88b2b39261f52ad6cd"}
Sep 19 05:42:11.478483 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:11.478440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:42:12.903466 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.903433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52bg8" event={"ID":"03a64404-88ed-4436-b000-10679862a70b","Type":"ContainerStarted","Data":"c671cb701ce149db62c7072421c48b4524841b1feb1f6d7d74e3abdfcf88e747"}
Sep 19 05:42:12.903894 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.903556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:42:12.905406 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.905381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnxxg" event={"ID":"6c42040f-8e4f-4d40-b199-58def6a92b99","Type":"ContainerStarted","Data":"338c0cedf5472e24e916b6d7c3414005bbd97c15949991245adf304599c4f59f"}
Sep 19 05:42:12.906864 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.906840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz" event={"ID":"05097b82-4166-466d-a5a7-d8cc8f600ecd","Type":"ContainerStarted","Data":"48839fbeffe193b907892cde9f4a1722f7fe06b8efe03ac8b1b899a3b5c71a27"}
Sep 19 05:42:12.907001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.906986    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:12.908479 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.908455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hq78s" event={"ID":"78f960ee-50d2-40db-a972-81da4ef0d346","Type":"ContainerStarted","Data":"f6fec5808bc23da78a7a0675d09cbdc8dd5104aba88ff0e85647ae9d28b88046"}
Sep 19 05:42:12.909069 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.909025    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz"
Sep 19 05:42:12.910652 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.910631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4" event={"ID":"eb3f381e-2699-4dbd-8e66-975e781514f3","Type":"ContainerStarted","Data":"f08669ab21fb0af544348afefecacfe9c4ad331e6e24b1eb3fc7a246edf59916"}
Sep 19 05:42:12.910746 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.910664    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4" event={"ID":"eb3f381e-2699-4dbd-8e66-975e781514f3","Type":"ContainerStarted","Data":"f26e2b13716741f6d1c62ba64374a98c183ad344c5f61e523a23589eccec8a05"}
Sep 19 05:42:12.910746 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.910681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4" event={"ID":"eb3f381e-2699-4dbd-8e66-975e781514f3","Type":"ContainerStarted","Data":"bcd785f392fcba62faf2bdcb5278c62aecb8b665088d74c709e4f86818283b5b"}
Sep 19 05:42:12.950700 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.950652    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-52bg8" podStartSLOduration=67.909662089 podStartE2EDuration="1m16.950639052s" totalImagesPullingTime="9.040976963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:12.949693849 +0000 UTC m=+78.560373834" watchObservedRunningTime="2026-09-19 05:42:12.950639052 +0000 UTC m=+78.561319027"
Sep 19 05:42:12.986209 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:12.986165    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855cf64699-sgwc4" podStartSLOduration=3.669831603 podStartE2EDuration="11.98615011s" totalImagesPullingTime="8.316318507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:12.985414205 +0000 UTC m=+78.596094169" watchObservedRunningTime="2026-09-19 05:42:12.98615011 +0000 UTC m=+78.596830071"
Sep 19 05:42:13.007453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:13.007410    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hq78s" podStartSLOduration=67.881045598 podStartE2EDuration="1m17.007396935s" totalImagesPullingTime="9.126351337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:13.00662509 +0000 UTC m=+78.617305071" watchObservedRunningTime="2026-09-19 05:42:13.007396935 +0000 UTC m=+78.618076926"
Sep 19 05:42:13.030595 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:13.030547    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cnxxg" podStartSLOduration=74.42749336 podStartE2EDuration="1m17.030528149s" totalImagesPullingTime="2.603034789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:13.029334086 +0000 UTC m=+78.640014052" watchObservedRunningTime="2026-09-19 05:42:13.030528149 +0000 UTC m=+78.641208115"
Sep 19 05:42:13.075485 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:13.075443    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c5657d594-jl9wz" podStartSLOduration=2.254891973 podStartE2EDuration="12.07542755s" totalImagesPullingTime="9.820535577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:13.065558577 +0000 UTC m=+78.676238563" watchObservedRunningTime="2026-09-19 05:42:13.07542755 +0000 UTC m=+78.686107513"
Sep 19 05:42:19.224526 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.224486    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zz8np"]
Sep 19 05:42:19.227503 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.227486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.230303 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.230278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 05:42:19.230438 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.230405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 05:42:19.230702 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.230674    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 05:42:19.231865 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.231741    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 05:42:19.232175 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.232156    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 05:42:19.232543 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.232055    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 05:42:19.233426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.233405    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x6wsd\""
Sep 19 05:42:19.335818 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.335786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-sys\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.335818 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.335827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-textfile\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.335846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-root\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.335899    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.335937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-wtmp\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.335984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z54d\" (UniqueName: \"kubernetes.io/projected/491e7ff6-0304-445d-af8a-fab61e96da6c-kube-api-access-7z54d\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.336009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491e7ff6-0304-445d-af8a-fab61e96da6c-metrics-client-ca\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336333 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.336105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.336333 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.336178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.436948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.436912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-wtmp\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437139 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.436957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z54d\" (UniqueName: \"kubernetes.io/projected/491e7ff6-0304-445d-af8a-fab61e96da6c-kube-api-access-7z54d\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437139 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.436978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491e7ff6-0304-445d-af8a-fab61e96da6c-metrics-client-ca\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437139 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.436994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437139 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437139 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-sys\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437134    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-wtmp\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-sys\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:19.437177    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-textfile\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-root\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:19.437258    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls podName:491e7ff6-0304-445d-af8a-fab61e96da6c nodeName:}" failed. No retries permitted until 2026-09-19 05:42:19.937236222 +0000 UTC m=+85.547916179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls") pod "node-exporter-zz8np" (UID: "491e7ff6-0304-445d-af8a-fab61e96da6c") : secret "node-exporter-tls" not found
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/491e7ff6-0304-445d-af8a-fab61e96da6c-root\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437370 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437717 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-textfile\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437760 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.437809 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.437792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491e7ff6-0304-445d-af8a-fab61e96da6c-metrics-client-ca\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.439609 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.439587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.446203 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.446183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z54d\" (UniqueName: \"kubernetes.io/projected/491e7ff6-0304-445d-af8a-fab61e96da6c-kube-api-access-7z54d\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.941693 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:19.941657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:19.941868 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:19.941807    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 05:42:19.941909 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:19.941879    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls podName:491e7ff6-0304-445d-af8a-fab61e96da6c nodeName:}" failed. No retries permitted until 2026-09-19 05:42:20.94186417 +0000 UTC m=+86.552544116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls") pod "node-exporter-zz8np" (UID: "491e7ff6-0304-445d-af8a-fab61e96da6c") : secret "node-exporter-tls" not found
Sep 19 05:42:20.949813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:20.949775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:20.952290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:20.952269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/491e7ff6-0304-445d-af8a-fab61e96da6c-node-exporter-tls\") pod \"node-exporter-zz8np\" (UID: \"491e7ff6-0304-445d-af8a-fab61e96da6c\") " pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:21.041391 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:21.041352    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zz8np"
Sep 19 05:42:21.054254 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:42:21.054227    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491e7ff6_0304_445d_af8a_fab61e96da6c.slice/crio-0fc95b2993dcd879fa0831c7907a8e06554aafa42caf4eef07d8517c77e28039 WatchSource:0}: Error finding container 0fc95b2993dcd879fa0831c7907a8e06554aafa42caf4eef07d8517c77e28039: Status 404 returned error can't find the container with id 0fc95b2993dcd879fa0831c7907a8e06554aafa42caf4eef07d8517c77e28039
Sep 19 05:42:21.943990 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:21.943939    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zz8np" event={"ID":"491e7ff6-0304-445d-af8a-fab61e96da6c","Type":"ContainerStarted","Data":"0fc95b2993dcd879fa0831c7907a8e06554aafa42caf4eef07d8517c77e28039"}
Sep 19 05:42:22.948654 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:22.948619    2581 generic.go:356] "Generic (PLEG): container finished" podID="491e7ff6-0304-445d-af8a-fab61e96da6c" containerID="a1a4d29dcb246b5dad33687133bade2e0202924638739fed8cc05a673027fdb9" exitCode=0
Sep 19 05:42:22.949069 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:22.948697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zz8np" event={"ID":"491e7ff6-0304-445d-af8a-fab61e96da6c","Type":"ContainerDied","Data":"a1a4d29dcb246b5dad33687133bade2e0202924638739fed8cc05a673027fdb9"}
Sep 19 05:42:23.954945 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:23.954903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zz8np" event={"ID":"491e7ff6-0304-445d-af8a-fab61e96da6c","Type":"ContainerStarted","Data":"a5d03a4fd536a017fb4f8347c14cee0f02788757157c6c3aac3b39c507c7a827"}
Sep 19 05:42:23.955456 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:23.955005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zz8np" event={"ID":"491e7ff6-0304-445d-af8a-fab61e96da6c","Type":"ContainerStarted","Data":"ec0e34ef76c3adfd914078defaa6022e69cc3349f89eef41574e4644f39b475d"}
Sep 19 05:42:23.992545 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:23.992497    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zz8np" podStartSLOduration=3.927451674 podStartE2EDuration="4.992482527s" totalImagesPullingTime="995.947183ms" totalInitContainerRuntime="69.08367ms" isStatefulPod=true podCreationTimestamp="2026-09-19 05:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:23.990710488 +0000 UTC m=+89.601390489" watchObservedRunningTime="2026-09-19 05:42:23.992482527 +0000 UTC m=+89.603162492"
Sep 19 05:42:24.865738 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:24.865711    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79657c785d-ljnjl"
Sep 19 05:42:26.490263 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.490216    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-959d65cbd-84n4l" podUID="d640860a-d9d8-4b07-a981-5a3e114ec318" containerName="registry" containerID="cri-o://b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380" gracePeriod=30
Sep 19 05:42:26.741515 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.741457    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:42:26.904375 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904341    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d640860a-d9d8-4b07-a981-5a3e114ec318-ca-trust-extracted\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904521 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-bound-sa-token\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904521 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904521 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904487    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-installation-pull-secrets\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904524    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-image-registry-private-configuration\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904658    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xt2j\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-kube-api-access-9xt2j\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904744 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-certificates\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.904744 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.904724    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-trusted-ca\") pod \"d640860a-d9d8-4b07-a981-5a3e114ec318\" (UID: \"d640860a-d9d8-4b07-a981-5a3e114ec318\") "
Sep 19 05:42:26.905368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.905286    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:42:26.905662 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.905588    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:42:26.907471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.907407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-kube-api-access-9xt2j" (OuterVolumeSpecName: "kube-api-access-9xt2j") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "kube-api-access-9xt2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:42:26.907471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.907410    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:42:26.907471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.907446    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:42:26.907703 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.907518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:42:26.907703 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.907533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:42:26.913546 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.913519    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d640860a-d9d8-4b07-a981-5a3e114ec318-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d640860a-d9d8-4b07-a981-5a3e114ec318" (UID: "d640860a-d9d8-4b07-a981-5a3e114ec318"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:42:26.967103 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.967071    2581 generic.go:356] "Generic (PLEG): container finished" podID="d640860a-d9d8-4b07-a981-5a3e114ec318" containerID="b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380" exitCode=0
Sep 19 05:42:26.967246 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.967113    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-959d65cbd-84n4l"
Sep 19 05:42:26.967246 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.967165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-959d65cbd-84n4l" event={"ID":"d640860a-d9d8-4b07-a981-5a3e114ec318","Type":"ContainerDied","Data":"b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380"}
Sep 19 05:42:26.967364 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.967254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-959d65cbd-84n4l" event={"ID":"d640860a-d9d8-4b07-a981-5a3e114ec318","Type":"ContainerDied","Data":"d56a73e8fe5cfc235043e482249655bd7e5d502933c2e3a3770f1eac213bc193"}
Sep 19 05:42:26.967364 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.967284    2581 scope.go:117] "RemoveContainer" containerID="b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380"
Sep 19 05:42:26.977782 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.977768    2581 scope.go:117] "RemoveContainer" containerID="b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380"
Sep 19 05:42:26.978463 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:42:26.978439    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380\": container with ID starting with b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380 not found: ID does not exist" containerID="b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380"
Sep 19 05:42:26.978551 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.978470    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380"} err="failed to get container status \"b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380\": rpc error: code = NotFound desc = could not find container \"b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380\": container with ID starting with b2f0d43a87a3590db5fa5e2bd7c52e2bd0f02b3127b99166f96b9d0480a76380 not found: ID does not exist"
Sep 19 05:42:26.994221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.994160    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-959d65cbd-84n4l"]
Sep 19 05:42:26.997003 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:26.996982    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-959d65cbd-84n4l"]
Sep 19 05:42:27.006274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006254    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xt2j\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-kube-api-access-9xt2j\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006275    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-certificates\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006287    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d640860a-d9d8-4b07-a981-5a3e114ec318-trusted-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006296    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d640860a-d9d8-4b07-a981-5a3e114ec318-ca-trust-extracted\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006304    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-bound-sa-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006312    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d640860a-d9d8-4b07-a981-5a3e114ec318-registry-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006320    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-installation-pull-secrets\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:27.006347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:27.006329    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d640860a-d9d8-4b07-a981-5a3e114ec318-image-registry-private-configuration\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:42:28.475583 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:28.475554    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d640860a-d9d8-4b07-a981-5a3e114ec318" path="/var/lib/kubelet/pods/d640860a-d9d8-4b07-a981-5a3e114ec318/volumes"
Sep 19 05:42:43.916429 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:43.916396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-52bg8"
Sep 19 05:42:58.078141 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:58.078111    2581 generic.go:356] "Generic (PLEG): container finished" podID="f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3" containerID="fbc6120e1007d9b023837875f13ed0ecd8a78c8cf0ee116abe63c0f5885f8e1f" exitCode=0
Sep 19 05:42:58.078509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:58.078192    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9" event={"ID":"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3","Type":"ContainerDied","Data":"fbc6120e1007d9b023837875f13ed0ecd8a78c8cf0ee116abe63c0f5885f8e1f"}
Sep 19 05:42:58.078509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:58.078493    2581 scope.go:117] "RemoveContainer" containerID="fbc6120e1007d9b023837875f13ed0ecd8a78c8cf0ee116abe63c0f5885f8e1f"
Sep 19 05:42:59.086260 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:42:59.086223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-svbh9" event={"ID":"f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3","Type":"ContainerStarted","Data":"93f36eb32ad564aca3a7ad60231ef2377427f54902db697fc033106e8c57bca0"}
Sep 19 05:45:21.229427 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.229337    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"]
Sep 19 05:45:21.229994 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.229731    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d640860a-d9d8-4b07-a981-5a3e114ec318" containerName="registry"
Sep 19 05:45:21.229994 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.229749    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d640860a-d9d8-4b07-a981-5a3e114ec318" containerName="registry"
Sep 19 05:45:21.229994 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.229807    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d640860a-d9d8-4b07-a981-5a3e114ec318" containerName="registry"
Sep 19 05:45:21.231634 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.231613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.235619 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.235599    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 05:45:21.235929 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.235912    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 05:45:21.235998 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.235962    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 05:45:21.236387 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.236372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s27qb\""
Sep 19 05:45:21.236704 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.236691    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 05:45:21.268692 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.268659    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"]
Sep 19 05:45:21.368089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.368023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e36ec9c-d443-4b99-ad4b-dd31ee998567-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.368089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.368087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9e36ec9c-d443-4b99-ad4b-dd31ee998567-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.368303 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.368134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft4qg\" (UniqueName: \"kubernetes.io/projected/9e36ec9c-d443-4b99-ad4b-dd31ee998567-kube-api-access-ft4qg\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.368303 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.368173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e36ec9c-d443-4b99-ad4b-dd31ee998567-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.368303 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.368206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9e36ec9c-d443-4b99-ad4b-dd31ee998567-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.469124 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.469090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft4qg\" (UniqueName: \"kubernetes.io/projected/9e36ec9c-d443-4b99-ad4b-dd31ee998567-kube-api-access-ft4qg\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.469292 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.469148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e36ec9c-d443-4b99-ad4b-dd31ee998567-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.469292 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.469183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9e36ec9c-d443-4b99-ad4b-dd31ee998567-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.469292 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.469237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e36ec9c-d443-4b99-ad4b-dd31ee998567-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.469292 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.469260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9e36ec9c-d443-4b99-ad4b-dd31ee998567-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.469575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.469554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9e36ec9c-d443-4b99-ad4b-dd31ee998567-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.471533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.471513    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e36ec9c-d443-4b99-ad4b-dd31ee998567-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.471762 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.471745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e36ec9c-d443-4b99-ad4b-dd31ee998567-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.471826 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.471785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9e36ec9c-d443-4b99-ad4b-dd31ee998567-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.480842 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.480787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft4qg\" (UniqueName: \"kubernetes.io/projected/9e36ec9c-d443-4b99-ad4b-dd31ee998567-kube-api-access-ft4qg\") pod \"opendatahub-operator-controller-manager-fd6794c5b-wsknt\" (UID: \"9e36ec9c-d443-4b99-ad4b-dd31ee998567\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.541471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.541440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:21.822235 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:21.822205    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"]
Sep 19 05:45:21.825521 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:45:21.825494    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e36ec9c_d443_4b99_ad4b_dd31ee998567.slice/crio-5ac4d8b5d1bda18cf44d035b13e2c1920a5dfde568619ac4237accc360462c99 WatchSource:0}: Error finding container 5ac4d8b5d1bda18cf44d035b13e2c1920a5dfde568619ac4237accc360462c99: Status 404 returned error can't find the container with id 5ac4d8b5d1bda18cf44d035b13e2c1920a5dfde568619ac4237accc360462c99
Sep 19 05:45:22.585654 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:22.585603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt" event={"ID":"9e36ec9c-d443-4b99-ad4b-dd31ee998567","Type":"ContainerStarted","Data":"5ac4d8b5d1bda18cf44d035b13e2c1920a5dfde568619ac4237accc360462c99"}
Sep 19 05:45:25.600340 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:25.600309    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt" event={"ID":"9e36ec9c-d443-4b99-ad4b-dd31ee998567","Type":"ContainerStarted","Data":"7c0e88ce42dabdac2afa119b2fd8600d8f756d41a9e5e988cc1737cecee180ce"}
Sep 19 05:45:26.605278 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:26.605241    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e36ec9c-d443-4b99-ad4b-dd31ee998567" containerID="7c0e88ce42dabdac2afa119b2fd8600d8f756d41a9e5e988cc1737cecee180ce" exitCode=0
Sep 19 05:45:26.605678 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:26.605330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt" event={"ID":"9e36ec9c-d443-4b99-ad4b-dd31ee998567","Type":"ContainerDied","Data":"7c0e88ce42dabdac2afa119b2fd8600d8f756d41a9e5e988cc1737cecee180ce"}
Sep 19 05:45:27.613075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:27.613018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt" event={"ID":"9e36ec9c-d443-4b99-ad4b-dd31ee998567","Type":"ContainerStarted","Data":"8f1a87fd281206346d3c8e5ad16b254c03fcbb144481f64a8f29652154534b9e"}
Sep 19 05:45:27.613543 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:27.613227    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:27.647533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:27.647485    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt" podStartSLOduration=2.3019048570000002 podStartE2EDuration="6.647456109s" totalImagesPullingTime="3.317221696s" totalInitContainerRuntime="1.028329556s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:27.645611071 +0000 UTC m=+273.256291057" watchObservedRunningTime="2026-09-19 05:45:27.647456109 +0000 UTC m=+273.258136072"
Sep 19 05:45:38.618740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:38.618711    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-wsknt"
Sep 19 05:45:47.694697 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.694661    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"]
Sep 19 05:45:47.700114 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.700094    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.706315 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.706295    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 19 05:45:47.707077 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.707059    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 19 05:45:47.729858 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.729833    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"]
Sep 19 05:45:47.769714 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.769688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78lvc\" (UniqueName: \"kubernetes.io/projected/6ff8af0d-3a8c-43b7-887c-41185725a0b4-kube-api-access-78lvc\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.769842 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.769728    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.769842 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.769763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tmp\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.870336 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.870302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78lvc\" (UniqueName: \"kubernetes.io/projected/6ff8af0d-3a8c-43b7-887c-41185725a0b4-kube-api-access-78lvc\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.870480 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.870346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.870480 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.870373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tmp\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.870480 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:45:47.870465    2581 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 19 05:45:47.870588 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:45:47.870550    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tls-certs podName:6ff8af0d-3a8c-43b7-887c-41185725a0b4 nodeName:}" failed. No retries permitted until 2026-09-19 05:45:48.370528292 +0000 UTC m=+293.981208237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tls-certs") pod "kube-auth-proxy-6b984b98c7-nfxsj" (UID: "6ff8af0d-3a8c-43b7-887c-41185725a0b4") : secret "kube-auth-proxy-tls" not found
Sep 19 05:45:47.872693 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.872672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tmp\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:47.880450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:47.880431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78lvc\" (UniqueName: \"kubernetes.io/projected/6ff8af0d-3a8c-43b7-887c-41185725a0b4-kube-api-access-78lvc\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:48.375426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:48.375390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:48.377957 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:48.377930    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ff8af0d-3a8c-43b7-887c-41185725a0b4-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-nfxsj\" (UID: \"6ff8af0d-3a8c-43b7-887c-41185725a0b4\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:48.608575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:48.608529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"
Sep 19 05:45:48.880585 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:48.880558    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj"]
Sep 19 05:45:48.882563 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:45:48.882532    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff8af0d_3a8c_43b7_887c_41185725a0b4.slice/crio-ddec12b1c04667a5c1a2e2a95b2613cccb8dfe794d1a586218f0855a6a2952fe WatchSource:0}: Error finding container ddec12b1c04667a5c1a2e2a95b2613cccb8dfe794d1a586218f0855a6a2952fe: Status 404 returned error can't find the container with id ddec12b1c04667a5c1a2e2a95b2613cccb8dfe794d1a586218f0855a6a2952fe
Sep 19 05:45:49.569818 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.569769    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"]
Sep 19 05:45:49.572126 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.572081    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.576004 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.575859    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 19 05:45:49.576004 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.575864    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gvr8h\""
Sep 19 05:45:49.576603 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.576351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 19 05:45:49.576603 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.576552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 05:45:49.576801 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.576782    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 19 05:45:49.613339 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.613277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"]
Sep 19 05:45:49.684020 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.683984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.684214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.684053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.684214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.684080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.684214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.684113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.684214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.684174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fffa4f7f-a50d-4819-b968-0e960362e33a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.684435 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.684252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.684435 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.684310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsv2v\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-kube-api-access-tsv2v\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.699373 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.699336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj" event={"ID":"6ff8af0d-3a8c-43b7-887c-41185725a0b4","Type":"ContainerStarted","Data":"ddec12b1c04667a5c1a2e2a95b2613cccb8dfe794d1a586218f0855a6a2952fe"}
Sep 19 05:45:49.784685 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.784869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsv2v\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-kube-api-access-tsv2v\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.784869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.784869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.784869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.784869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.785154 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.784894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fffa4f7f-a50d-4819-b968-0e960362e33a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.785431 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.785377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.788330 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.788007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.788879 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.788852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.790504 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.790472    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fffa4f7f-a50d-4819-b968-0e960362e33a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.790579 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.790528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.794978 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.794953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsv2v\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-kube-api-access-tsv2v\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.795112 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.795090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-c7wh7\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:49.885160 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:49.885075    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:50.180457 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:50.180260    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"]
Sep 19 05:45:50.183526 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:45:50.183496    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfffa4f7f_a50d_4819_b968_0e960362e33a.slice/crio-584003da327f46d93e87fa47c4ff67e383bc7c70e771ee062ed258c6d48af704 WatchSource:0}: Error finding container 584003da327f46d93e87fa47c4ff67e383bc7c70e771ee062ed258c6d48af704: Status 404 returned error can't find the container with id 584003da327f46d93e87fa47c4ff67e383bc7c70e771ee062ed258c6d48af704
Sep 19 05:45:50.705277 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:50.705242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" event={"ID":"fffa4f7f-a50d-4819-b968-0e960362e33a","Type":"ContainerStarted","Data":"584003da327f46d93e87fa47c4ff67e383bc7c70e771ee062ed258c6d48af704"}
Sep 19 05:45:51.712359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:51.712287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj" event={"ID":"6ff8af0d-3a8c-43b7-887c-41185725a0b4","Type":"ContainerStarted","Data":"678627e7d618ff99aa44acc1accfa1de9389794a7d9df68534eac0aec70982d5"}
Sep 19 05:45:51.737160 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:51.737076    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-nfxsj" podStartSLOduration=2.026322033 podStartE2EDuration="4.737055637s" totalImagesPullingTime="2.710733604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:51.736183956 +0000 UTC m=+297.346863961" watchObservedRunningTime="2026-09-19 05:45:51.737055637 +0000 UTC m=+297.347735605"
Sep 19 05:45:53.426265 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:53.426221    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:45:53.426577 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:53.426311    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:45:53.723595 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:53.723498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" event={"ID":"fffa4f7f-a50d-4819-b968-0e960362e33a","Type":"ContainerStarted","Data":"4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4"}
Sep 19 05:45:53.723759 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:53.723604    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:45:53.749970 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:53.749923    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" podStartSLOduration=1.510674253 podStartE2EDuration="4.749905545s" totalImagesPullingTime="3.239231292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:53.748187009 +0000 UTC m=+299.358867001" watchObservedRunningTime="2026-09-19 05:45:53.749905545 +0000 UTC m=+299.360585509"
Sep 19 05:45:54.732056 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:54.731988    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-c7wh7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 05:45:54.732535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:54.732076    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" podUID="fffa4f7f-a50d-4819-b968-0e960362e33a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:45:56.368748 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:56.368719    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:45:56.371155 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:56.371136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:45:56.374220 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:56.374203    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:45:56.376468 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:56.376448    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:45:56.379953 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:56.379934    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 05:45:57.731492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:45:57.731459    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:46:04.294910 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.294874    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"]
Sep 19 05:46:04.298184 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.298162    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.302227 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.302205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5zzk2\""
Sep 19 05:46:04.303562 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.303542    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 19 05:46:04.314718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.314692    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"]
Sep 19 05:46:04.322097 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.322072    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"]
Sep 19 05:46:04.325223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.325203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:04.327915 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.327895    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-b87hw\""
Sep 19 05:46:04.328230 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.328213    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 19 05:46:04.344663 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.344635    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"]
Sep 19 05:46:04.414850 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.414817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:04.415006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.414855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmnz\" (UniqueName: \"kubernetes.io/projected/1077e5d4-7ffe-401e-a3f4-bf3ae4916663-kube-api-access-wrmnz\") pod \"kserve-controller-manager-d6f4b65b8-p4q72\" (UID: \"1077e5d4-7ffe-401e-a3f4-bf3ae4916663\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.415006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.414893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1077e5d4-7ffe-401e-a3f4-bf3ae4916663-cert\") pod \"kserve-controller-manager-d6f4b65b8-p4q72\" (UID: \"1077e5d4-7ffe-401e-a3f4-bf3ae4916663\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.415006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.414951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwshs\" (UniqueName: \"kubernetes.io/projected/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-kube-api-access-xwshs\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:04.515771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.515735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1077e5d4-7ffe-401e-a3f4-bf3ae4916663-cert\") pod \"kserve-controller-manager-d6f4b65b8-p4q72\" (UID: \"1077e5d4-7ffe-401e-a3f4-bf3ae4916663\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.515771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.515773    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwshs\" (UniqueName: \"kubernetes.io/projected/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-kube-api-access-xwshs\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:04.516062 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.515819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:04.516062 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.515848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmnz\" (UniqueName: \"kubernetes.io/projected/1077e5d4-7ffe-401e-a3f4-bf3ae4916663-kube-api-access-wrmnz\") pod \"kserve-controller-manager-d6f4b65b8-p4q72\" (UID: \"1077e5d4-7ffe-401e-a3f4-bf3ae4916663\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.516062 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:46:04.515966    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 19 05:46:04.516062 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:46:04.516024    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-cert podName:14dc59d9-81f4-44d4-81d3-d4cf52cb08fc nodeName:}" failed. No retries permitted until 2026-09-19 05:46:05.016008504 +0000 UTC m=+310.626688446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-cert") pod "llmisvc-controller-manager-777c6bd7d5-px6dd" (UID: "14dc59d9-81f4-44d4-81d3-d4cf52cb08fc") : secret "llmisvc-webhook-server-cert" not found
Sep 19 05:46:04.518408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.518384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1077e5d4-7ffe-401e-a3f4-bf3ae4916663-cert\") pod \"kserve-controller-manager-d6f4b65b8-p4q72\" (UID: \"1077e5d4-7ffe-401e-a3f4-bf3ae4916663\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.532701 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.532673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwshs\" (UniqueName: \"kubernetes.io/projected/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-kube-api-access-xwshs\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:04.533911 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.533885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmnz\" (UniqueName: \"kubernetes.io/projected/1077e5d4-7ffe-401e-a3f4-bf3ae4916663-kube-api-access-wrmnz\") pod \"kserve-controller-manager-d6f4b65b8-p4q72\" (UID: \"1077e5d4-7ffe-401e-a3f4-bf3ae4916663\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.609790 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.609711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:04.898948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.898924    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:46:04.899245 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:04.899217    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"]
Sep 19 05:46:05.019683 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:05.019650    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:05.022299 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:05.022278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14dc59d9-81f4-44d4-81d3-d4cf52cb08fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-px6dd\" (UID: \"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:05.235092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:05.234985    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:05.521565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:05.521490    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"]
Sep 19 05:46:05.525315 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:46:05.525282    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod14dc59d9_81f4_44d4_81d3_d4cf52cb08fc.slice/crio-7829dfa318eae79747d01aab1d7141fcf323c03782daf754dd112f8cd09f14e1 WatchSource:0}: Error finding container 7829dfa318eae79747d01aab1d7141fcf323c03782daf754dd112f8cd09f14e1: Status 404 returned error can't find the container with id 7829dfa318eae79747d01aab1d7141fcf323c03782daf754dd112f8cd09f14e1
Sep 19 05:46:05.772515 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:05.772414    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72" event={"ID":"1077e5d4-7ffe-401e-a3f4-bf3ae4916663","Type":"ContainerStarted","Data":"0d45004c42b70d55d120356a3b4037bf5a295162e472fb984aa4499a1c726a4c"}
Sep 19 05:46:05.773839 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:05.773811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd" event={"ID":"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc","Type":"ContainerStarted","Data":"7829dfa318eae79747d01aab1d7141fcf323c03782daf754dd112f8cd09f14e1"}
Sep 19 05:46:07.786565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:07.786522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72" event={"ID":"1077e5d4-7ffe-401e-a3f4-bf3ae4916663","Type":"ContainerStarted","Data":"848d38a3f824ac58d2796360059f634ae38387074450ee977c3dfa3850bef2e7"}
Sep 19 05:46:07.786968 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:07.786611    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:07.807731 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:07.807678    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72" podStartSLOduration=1.475773305 podStartE2EDuration="3.807664136s" totalImagesPullingTime="2.331890831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:46:07.806253517 +0000 UTC m=+313.416933506" watchObservedRunningTime="2026-09-19 05:46:07.807664136 +0000 UTC m=+313.418344099"
Sep 19 05:46:08.128351 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.128280    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"]
Sep 19 05:46:08.128514 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.128488    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" podUID="fffa4f7f-a50d-4819-b968-0e960362e33a" containerName="discovery" containerID="cri-o://4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4" gracePeriod=30
Sep 19 05:46:08.752243 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.752217    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:46:08.796895 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.796852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd" event={"ID":"14dc59d9-81f4-44d4-81d3-d4cf52cb08fc","Type":"ContainerStarted","Data":"7eb4f8489d32cc0ffd58775a31827a6873d8aa7a931045922ecf0a4d407bd847"}
Sep 19 05:46:08.797337 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.796937    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:46:08.798252 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.798230    2581 generic.go:356] "Generic (PLEG): container finished" podID="fffa4f7f-a50d-4819-b968-0e960362e33a" containerID="4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4" exitCode=0
Sep 19 05:46:08.798368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.798268    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"
Sep 19 05:46:08.798368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.798315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" event={"ID":"fffa4f7f-a50d-4819-b968-0e960362e33a","Type":"ContainerDied","Data":"4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4"}
Sep 19 05:46:08.798368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.798351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7" event={"ID":"fffa4f7f-a50d-4819-b968-0e960362e33a","Type":"ContainerDied","Data":"584003da327f46d93e87fa47c4ff67e383bc7c70e771ee062ed258c6d48af704"}
Sep 19 05:46:08.798484 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.798370    2581 scope.go:117] "RemoveContainer" containerID="4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4"
Sep 19 05:46:08.806974 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.806957    2581 scope.go:117] "RemoveContainer" containerID="4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4"
Sep 19 05:46:08.807256 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:46:08.807240    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4\": container with ID starting with 4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4 not found: ID does not exist" containerID="4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4"
Sep 19 05:46:08.807310 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.807264    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4"} err="failed to get container status \"4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4\": rpc error: code = NotFound desc = could not find container \"4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4\": container with ID starting with 4574b0530da77b54c86c00a01f21dc1bb76fc0134fcc3cb4114be01ebb0f3fa4 not found: ID does not exist"
Sep 19 05:46:08.820403 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.820349    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd" podStartSLOduration=1.688711336 podStartE2EDuration="4.820331155s" totalImagesPullingTime="3.131619819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:46:08.819461582 +0000 UTC m=+314.430141555" watchObservedRunningTime="2026-09-19 05:46:08.820331155 +0000 UTC m=+314.431011156"
Sep 19 05:46:08.856334 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856299    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fffa4f7f-a50d-4819-b968-0e960362e33a-local-certs\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856334 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856340    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-kubeconfig\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856528 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-ca-configmap\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856528 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-dns-cert\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856528 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856471    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-cacerts\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856528 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856505    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-token\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856721 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856529    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsv2v\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-kube-api-access-tsv2v\") pod \"fffa4f7f-a50d-4819-b968-0e960362e33a\" (UID: \"fffa4f7f-a50d-4819-b968-0e960362e33a\") "
Sep 19 05:46:08.856899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.856867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:46:08.859004 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.858904    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-token" (OuterVolumeSpecName: "istio-token") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:46:08.859135 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.858998    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fffa4f7f-a50d-4819-b968-0e960362e33a-local-certs" (OuterVolumeSpecName: "local-certs") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:46:08.859176 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.859140    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:46:08.859176 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.859159    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:46:08.859363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.859337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-kube-api-access-tsv2v" (OuterVolumeSpecName: "kube-api-access-tsv2v") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "kube-api-access-tsv2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:46:08.859363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.859349    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-cacerts" (OuterVolumeSpecName: "cacerts") pod "fffa4f7f-a50d-4819-b968-0e960362e33a" (UID: "fffa4f7f-a50d-4819-b968-0e960362e33a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:46:08.958149 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958104    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-cacerts\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:08.958149 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958144    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:08.958149 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958161    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsv2v\" (UniqueName: \"kubernetes.io/projected/fffa4f7f-a50d-4819-b968-0e960362e33a-kube-api-access-tsv2v\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:08.958450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958175    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fffa4f7f-a50d-4819-b968-0e960362e33a-local-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:08.958450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958189    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-kubeconfig\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:08.958450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958201    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-ca-configmap\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:08.958450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:08.958214    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fffa4f7f-a50d-4819-b968-0e960362e33a-istio-csr-dns-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:46:09.121799 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:09.121728    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"]
Sep 19 05:46:09.125977 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:09.125951    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c7wh7"]
Sep 19 05:46:10.476496 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:10.476460    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fffa4f7f-a50d-4819-b968-0e960362e33a" path="/var/lib/kubelet/pods/fffa4f7f-a50d-4819-b968-0e960362e33a/volumes"
Sep 19 05:46:38.803909 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:38.803818    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-p4q72"
Sep 19 05:46:39.806120 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:46:39.806088    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-px6dd"
Sep 19 05:47:12.299004 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.298967    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"]
Sep 19 05:47:12.299453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.299264    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fffa4f7f-a50d-4819-b968-0e960362e33a" containerName="discovery"
Sep 19 05:47:12.299453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.299279    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fffa4f7f-a50d-4819-b968-0e960362e33a" containerName="discovery"
Sep 19 05:47:12.299453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.299344    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fffa4f7f-a50d-4819-b968-0e960362e33a" containerName="discovery"
Sep 19 05:47:12.301096 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.301080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:12.309231 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.309192    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 05:47:12.309231 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.309214    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bl4s4\""
Sep 19 05:47:12.309409 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.309247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 05:47:12.324647 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.324615    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"]
Sep 19 05:47:12.452347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.452311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vbdr\" (UniqueName: \"kubernetes.io/projected/26ec8ea8-6a64-4509-8f76-2706900b8b14-kube-api-access-9vbdr\") pod \"limitador-operator-controller-manager-74495c69f8-66lvg\" (UID: \"26ec8ea8-6a64-4509-8f76-2706900b8b14\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:12.553532 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.553446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vbdr\" (UniqueName: \"kubernetes.io/projected/26ec8ea8-6a64-4509-8f76-2706900b8b14-kube-api-access-9vbdr\") pod \"limitador-operator-controller-manager-74495c69f8-66lvg\" (UID: \"26ec8ea8-6a64-4509-8f76-2706900b8b14\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:12.569611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.569588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vbdr\" (UniqueName: \"kubernetes.io/projected/26ec8ea8-6a64-4509-8f76-2706900b8b14-kube-api-access-9vbdr\") pod \"limitador-operator-controller-manager-74495c69f8-66lvg\" (UID: \"26ec8ea8-6a64-4509-8f76-2706900b8b14\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:12.610534 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.610510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:12.895102 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:12.894992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"]
Sep 19 05:47:12.898075 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:12.898022    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ec8ea8_6a64_4509_8f76_2706900b8b14.slice/crio-ad83e0e6abec197837ccf9677537094cacc79fd5a3f0ebd5beb63b6aad0a370f WatchSource:0}: Error finding container ad83e0e6abec197837ccf9677537094cacc79fd5a3f0ebd5beb63b6aad0a370f: Status 404 returned error can't find the container with id ad83e0e6abec197837ccf9677537094cacc79fd5a3f0ebd5beb63b6aad0a370f
Sep 19 05:47:13.058499 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:13.058470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg" event={"ID":"26ec8ea8-6a64-4509-8f76-2706900b8b14","Type":"ContainerStarted","Data":"ad83e0e6abec197837ccf9677537094cacc79fd5a3f0ebd5beb63b6aad0a370f"}
Sep 19 05:47:14.794731 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:14.794698    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q2v9j"]
Sep 19 05:47:14.800956 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:14.800932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:14.806016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:14.805994    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zqmmf\""
Sep 19 05:47:14.819910 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:14.819884    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q2v9j"]
Sep 19 05:47:14.972202 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:14.972160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dssf\" (UniqueName: \"kubernetes.io/projected/e229b570-bffc-4161-ae3b-161067cde1ef-kube-api-access-8dssf\") pod \"authorino-operator-76b478bd78-q2v9j\" (UID: \"e229b570-bffc-4161-ae3b-161067cde1ef\") " pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:15.072988 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:15.072908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dssf\" (UniqueName: \"kubernetes.io/projected/e229b570-bffc-4161-ae3b-161067cde1ef-kube-api-access-8dssf\") pod \"authorino-operator-76b478bd78-q2v9j\" (UID: \"e229b570-bffc-4161-ae3b-161067cde1ef\") " pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:15.102729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:15.102698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dssf\" (UniqueName: \"kubernetes.io/projected/e229b570-bffc-4161-ae3b-161067cde1ef-kube-api-access-8dssf\") pod \"authorino-operator-76b478bd78-q2v9j\" (UID: \"e229b570-bffc-4161-ae3b-161067cde1ef\") " pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:15.113837 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:15.113810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:15.585617 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:15.585585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q2v9j"]
Sep 19 05:47:15.588979 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:15.588948    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode229b570_bffc_4161_ae3b_161067cde1ef.slice/crio-d71810ef9f063d1f6ace7db1b43606928e7b5f917b1adb968d986248a448127a WatchSource:0}: Error finding container d71810ef9f063d1f6ace7db1b43606928e7b5f917b1adb968d986248a448127a: Status 404 returned error can't find the container with id d71810ef9f063d1f6ace7db1b43606928e7b5f917b1adb968d986248a448127a
Sep 19 05:47:16.075330 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:16.075290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg" event={"ID":"26ec8ea8-6a64-4509-8f76-2706900b8b14","Type":"ContainerStarted","Data":"2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43"}
Sep 19 05:47:16.075793 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:16.075370    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:16.076713 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:16.076689    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j" event={"ID":"e229b570-bffc-4161-ae3b-161067cde1ef","Type":"ContainerStarted","Data":"d71810ef9f063d1f6ace7db1b43606928e7b5f917b1adb968d986248a448127a"}
Sep 19 05:47:16.099978 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:16.099922    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg" podStartSLOduration=1.640434039 podStartE2EDuration="4.099907644s" totalImagesPullingTime="2.459473605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:16.099021216 +0000 UTC m=+381.709701200" watchObservedRunningTime="2026-09-19 05:47:16.099907644 +0000 UTC m=+381.710587608"
Sep 19 05:47:17.082393 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.082354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j" event={"ID":"e229b570-bffc-4161-ae3b-161067cde1ef","Type":"ContainerStarted","Data":"67dea3031755bdbbb6cf3a13d0cf26406ea205a1de777cee68935c71f718df4a"}
Sep 19 05:47:17.135299 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.135223    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"]
Sep 19 05:47:17.138306 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.138288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:17.145008 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:17.144974    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-operator-controller-manager-dockercfg-schvr\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-schvr\"" type="*v1.Secret"
Sep 19 05:47:17.147120 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:17.147097    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-operator-controller-env\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" type="*v1.ConfigMap"
Sep 19 05:47:17.220525 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.220494    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"]
Sep 19 05:47:17.291349 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.291317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4-kube-api-access-5wcs9\") pod \"dns-operator-controller-manager-7b84cd7c9c-hr2fp\" (UID: \"3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:17.383527 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.383472    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j" podStartSLOduration=2.127786157 podStartE2EDuration="3.383456692s" totalImagesPullingTime="1.255670535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:17.282025457 +0000 UTC m=+382.892705444" watchObservedRunningTime="2026-09-19 05:47:17.383456692 +0000 UTC m=+382.994136656"
Sep 19 05:47:17.392382 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.392314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4-kube-api-access-5wcs9\") pod \"dns-operator-controller-manager-7b84cd7c9c-hr2fp\" (UID: \"3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:17.409856 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:17.409823    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wcs9\" (UniqueName: \"kubernetes.io/projected/3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4-kube-api-access-5wcs9\") pod \"dns-operator-controller-manager-7b84cd7c9c-hr2fp\" (UID: \"3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:18.065044 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:18.065013    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 19 05:47:18.086932 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:18.086907    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:18.261639 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:18.261606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-schvr\""
Sep 19 05:47:18.268869 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:18.268850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:18.549572 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:18.549548    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"]
Sep 19 05:47:18.552121 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:18.552095    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eaa1d0a_eaef_4bd3_8a40_3d8db9e651b4.slice/crio-8232c28ec30c06913cc3c867559ffc0b2f28ee08f688f65b7dcb961a557b9f5c WatchSource:0}: Error finding container 8232c28ec30c06913cc3c867559ffc0b2f28ee08f688f65b7dcb961a557b9f5c: Status 404 returned error can't find the container with id 8232c28ec30c06913cc3c867559ffc0b2f28ee08f688f65b7dcb961a557b9f5c
Sep 19 05:47:19.092589 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:19.092551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp" event={"ID":"3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4","Type":"ContainerStarted","Data":"8232c28ec30c06913cc3c867559ffc0b2f28ee08f688f65b7dcb961a557b9f5c"}
Sep 19 05:47:21.102579 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.102544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp" event={"ID":"3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4","Type":"ContainerStarted","Data":"8091db31ef519c6c0d38d0e281c4ddf10a7ea253a32d8dd2411ced479e3bf30c"}
Sep 19 05:47:21.103028 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.102692    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:21.134250 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.134203    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp" podStartSLOduration=2.560215216 podStartE2EDuration="4.134190567s" totalImagesPullingTime="1.573975351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:21.132967702 +0000 UTC m=+386.743647693" watchObservedRunningTime="2026-09-19 05:47:21.134190567 +0000 UTC m=+386.744870530"
Sep 19 05:47:21.628709 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.628672    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"]
Sep 19 05:47:21.632083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.632067    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.635722 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.635702    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tssrv\""
Sep 19 05:47:21.653295 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.653270    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"]
Sep 19 05:47:21.729678 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.729648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92fa6304-a769-4fe2-8e46-c08a1975e4c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.729678 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.729682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5xfp\" (UniqueName: \"kubernetes.io/projected/92fa6304-a769-4fe2-8e46-c08a1975e4c8-kube-api-access-f5xfp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.830937 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.830905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92fa6304-a769-4fe2-8e46-c08a1975e4c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.831128 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.830941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5xfp\" (UniqueName: \"kubernetes.io/projected/92fa6304-a769-4fe2-8e46-c08a1975e4c8-kube-api-access-f5xfp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.831337 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.831313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92fa6304-a769-4fe2-8e46-c08a1975e4c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.844185 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.844165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5xfp\" (UniqueName: \"kubernetes.io/projected/92fa6304-a769-4fe2-8e46-c08a1975e4c8-kube-api-access-f5xfp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:21.942547 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:21.942475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:22.249359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:22.249228    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"]
Sep 19 05:47:22.252304 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:22.252279    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fa6304_a769_4fe2_8e46_c08a1975e4c8.slice/crio-3060c0f05a8471dd3c50c0a3fe6008f1fc9287a1a5b759299438d613f7825a03 WatchSource:0}: Error finding container 3060c0f05a8471dd3c50c0a3fe6008f1fc9287a1a5b759299438d613f7825a03: Status 404 returned error can't find the container with id 3060c0f05a8471dd3c50c0a3fe6008f1fc9287a1a5b759299438d613f7825a03
Sep 19 05:47:23.113102 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:23.113067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq" event={"ID":"92fa6304-a769-4fe2-8e46-c08a1975e4c8","Type":"ContainerStarted","Data":"3060c0f05a8471dd3c50c0a3fe6008f1fc9287a1a5b759299438d613f7825a03"}
Sep 19 05:47:26.128400 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:26.128321    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq" event={"ID":"92fa6304-a769-4fe2-8e46-c08a1975e4c8","Type":"ContainerStarted","Data":"88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5"}
Sep 19 05:47:26.128744 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:26.128434    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:26.151806 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:26.151743    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq" podStartSLOduration=1.5835452719999998 podStartE2EDuration="5.151727159s" totalImagesPullingTime="3.568181887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:26.150301609 +0000 UTC m=+391.760981605" watchObservedRunningTime="2026-09-19 05:47:26.151727159 +0000 UTC m=+391.762407122"
Sep 19 05:47:27.085012 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:27.084985    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:28.610028 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.609993    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"]
Sep 19 05:47:28.613464 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.613443    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.616007 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.615988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jgws2\""
Sep 19 05:47:28.616124 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.616028    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 19 05:47:28.616124 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.616056    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 19 05:47:28.622102 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.622082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"]
Sep 19 05:47:28.691354 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.691317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.691482 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.691400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.691482 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.691420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpwwl\" (UniqueName: \"kubernetes.io/projected/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-kube-api-access-wpwwl\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.792392 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.792357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.792392 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.792392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpwwl\" (UniqueName: \"kubernetes.io/projected/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-kube-api-access-wpwwl\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.792587 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.792416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.792587 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:28.792504    2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 19 05:47:28.792587 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:28.792584    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-plugin-serving-cert podName:f2592f39-d2fe-4de3-85d7-8d6fed48b5fb nodeName:}" failed. No retries permitted until 2026-09-19 05:47:29.292567312 +0000 UTC m=+394.903247271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-wh695" (UID: "f2592f39-d2fe-4de3-85d7-8d6fed48b5fb") : secret "plugin-serving-cert" not found
Sep 19 05:47:28.793124 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.793107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:28.809263 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:28.809233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpwwl\" (UniqueName: \"kubernetes.io/projected/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-kube-api-access-wpwwl\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:29.095389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:29.095353    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-q2v9j"
Sep 19 05:47:29.296510 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:29.296477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:29.298995 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:29.298977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2592f39-d2fe-4de3-85d7-8d6fed48b5fb-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-wh695\" (UID: \"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:29.523350 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:29.523267    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"
Sep 19 05:47:29.799922 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:29.799899    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695"]
Sep 19 05:47:29.802455 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:29.802430    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2592f39_d2fe_4de3_85d7_8d6fed48b5fb.slice/crio-4b6c56d25b40a8e95c69c1c1872e0c1eb635f32672ce532f59c9c067e7dbf0c8 WatchSource:0}: Error finding container 4b6c56d25b40a8e95c69c1c1872e0c1eb635f32672ce532f59c9c067e7dbf0c8: Status 404 returned error can't find the container with id 4b6c56d25b40a8e95c69c1c1872e0c1eb635f32672ce532f59c9c067e7dbf0c8
Sep 19 05:47:30.147918 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:30.147832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695" event={"ID":"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb","Type":"ContainerStarted","Data":"4b6c56d25b40a8e95c69c1c1872e0c1eb635f32672ce532f59c9c067e7dbf0c8"}
Sep 19 05:47:32.110067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:32.109730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hr2fp"
Sep 19 05:47:35.174017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:35.173937    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695" event={"ID":"f2592f39-d2fe-4de3-85d7-8d6fed48b5fb","Type":"ContainerStarted","Data":"09e9017a896501c025424062bb4523c51fbd0141195fca7334f313b70c701e36"}
Sep 19 05:47:35.213103 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:35.213053    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-wh695" podStartSLOduration=2.58623832 podStartE2EDuration="7.213022704s" totalImagesPullingTime="4.626784384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:35.21151848 +0000 UTC m=+400.822198443" watchObservedRunningTime="2026-09-19 05:47:35.213022704 +0000 UTC m=+400.823702668"
Sep 19 05:47:37.135929 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:37.135895    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:39.782959 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.782916    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"]
Sep 19 05:47:39.783400 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.783139    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" containerName="manager" containerID="cri-o://88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5" gracePeriod=2
Sep 19 05:47:39.788257 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.788232    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"]
Sep 19 05:47:39.811073 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.811044    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"]
Sep 19 05:47:39.811281 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.811257    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" containerName="manager" containerID="cri-o://2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43" gracePeriod=2
Sep 19 05:47:39.830971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.830943    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"]
Sep 19 05:47:39.843622 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:39.843583    2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"26ec8ea8-6a64-4509-8f76-2706900b8b14\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-66lvg\": pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:39.848557 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:39.848524    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:39.860830 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.860806    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"]
Sep 19 05:47:39.861455 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.861428    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" containerName="manager"
Sep 19 05:47:39.861539 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.861459    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" containerName="manager"
Sep 19 05:47:39.861539 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.861477    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" containerName="manager"
Sep 19 05:47:39.861539 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.861487    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" containerName="manager"
Sep 19 05:47:39.861646 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.861593    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" containerName="manager"
Sep 19 05:47:39.861646 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.861603    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" containerName="manager"
Sep 19 05:47:39.864648 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.864628    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"]
Sep 19 05:47:39.864788 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.864768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:39.867757 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.867740    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:39.871247 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:39.871223    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:39.873430 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:39.873401    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:39.882858 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.882833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t9jx8\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:39.883005 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.882989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26pxv\" (UniqueName: \"kubernetes.io/projected/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-kube-api-access-26pxv\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t9jx8\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:39.885806 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.885785    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"]
Sep 19 05:47:39.896525 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.896500    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"]
Sep 19 05:47:39.953516 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:39.953471    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:39.956650 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:39.956623    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:39.984024 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.983543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t9jx8\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:39.984024 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.983599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26pxv\" (UniqueName: \"kubernetes.io/projected/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-kube-api-access-26pxv\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t9jx8\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:39.984024 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.983640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg2zw\" (UniqueName: \"kubernetes.io/projected/d9826c18-e90d-48b2-b98b-ea68840ffcc8-kube-api-access-xg2zw\") pod \"limitador-operator-controller-manager-74495c69f8-jjtpw\" (UID: \"d9826c18-e90d-48b2-b98b-ea68840ffcc8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:39.984024 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.983900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t9jx8\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:39.999661 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:39.999599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26pxv\" (UniqueName: \"kubernetes.io/projected/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-kube-api-access-26pxv\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t9jx8\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:40.045225 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.045200    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.048011 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.047981    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.050132 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.050103    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.052714 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.052696    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.055816 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.055795    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.058196 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.058172    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.084650 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.084626    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92fa6304-a769-4fe2-8e46-c08a1975e4c8-extensions-socket-volume\") pod \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") "
Sep 19 05:47:40.084739 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.084670    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vbdr\" (UniqueName: \"kubernetes.io/projected/26ec8ea8-6a64-4509-8f76-2706900b8b14-kube-api-access-9vbdr\") pod \"26ec8ea8-6a64-4509-8f76-2706900b8b14\" (UID: \"26ec8ea8-6a64-4509-8f76-2706900b8b14\") "
Sep 19 05:47:40.084739 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.084688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5xfp\" (UniqueName: \"kubernetes.io/projected/92fa6304-a769-4fe2-8e46-c08a1975e4c8-kube-api-access-f5xfp\") pod \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\" (UID: \"92fa6304-a769-4fe2-8e46-c08a1975e4c8\") "
Sep 19 05:47:40.084813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.084766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg2zw\" (UniqueName: \"kubernetes.io/projected/d9826c18-e90d-48b2-b98b-ea68840ffcc8-kube-api-access-xg2zw\") pod \"limitador-operator-controller-manager-74495c69f8-jjtpw\" (UID: \"d9826c18-e90d-48b2-b98b-ea68840ffcc8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:40.085123 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.085097    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92fa6304-a769-4fe2-8e46-c08a1975e4c8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "92fa6304-a769-4fe2-8e46-c08a1975e4c8" (UID: "92fa6304-a769-4fe2-8e46-c08a1975e4c8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:40.086932 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.086906    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26ec8ea8-6a64-4509-8f76-2706900b8b14-kube-api-access-9vbdr" (OuterVolumeSpecName: "kube-api-access-9vbdr") pod "26ec8ea8-6a64-4509-8f76-2706900b8b14" (UID: "26ec8ea8-6a64-4509-8f76-2706900b8b14"). InnerVolumeSpecName "kube-api-access-9vbdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:40.087000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.086933    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92fa6304-a769-4fe2-8e46-c08a1975e4c8-kube-api-access-f5xfp" (OuterVolumeSpecName: "kube-api-access-f5xfp") pod "92fa6304-a769-4fe2-8e46-c08a1975e4c8" (UID: "92fa6304-a769-4fe2-8e46-c08a1975e4c8"). InnerVolumeSpecName "kube-api-access-f5xfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:40.102319 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.102297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg2zw\" (UniqueName: \"kubernetes.io/projected/d9826c18-e90d-48b2-b98b-ea68840ffcc8-kube-api-access-xg2zw\") pod \"limitador-operator-controller-manager-74495c69f8-jjtpw\" (UID: \"d9826c18-e90d-48b2-b98b-ea68840ffcc8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:40.185508 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.185478    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92fa6304-a769-4fe2-8e46-c08a1975e4c8-extensions-socket-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:47:40.185508 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.185504    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vbdr\" (UniqueName: \"kubernetes.io/projected/26ec8ea8-6a64-4509-8f76-2706900b8b14-kube-api-access-9vbdr\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:47:40.185508 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.185513    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5xfp\" (UniqueName: \"kubernetes.io/projected/92fa6304-a769-4fe2-8e46-c08a1975e4c8-kube-api-access-f5xfp\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:47:40.195433 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.195403    2581 generic.go:356] "Generic (PLEG): container finished" podID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" containerID="88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5" exitCode=0
Sep 19 05:47:40.195546 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.195446    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.195546 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.195504    2581 scope.go:117] "RemoveContainer" containerID="88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5"
Sep 19 05:47:40.197001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.196979    2581 generic.go:356] "Generic (PLEG): container finished" podID="26ec8ea8-6a64-4509-8f76-2706900b8b14" containerID="2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43" exitCode=0
Sep 19 05:47:40.197131 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.197056    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.200498 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.200463    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.202770 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.202743    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.203859 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.203837    2581 scope.go:117] "RemoveContainer" containerID="88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5"
Sep 19 05:47:40.204170 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.204151    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5\": container with ID starting with 88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5 not found: ID does not exist" containerID="88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5"
Sep 19 05:47:40.204260 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.204180    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5"} err="failed to get container status \"88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5\": rpc error: code = NotFound desc = could not find container \"88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5\": container with ID starting with 88f4b1d4303656c6b583c8709b862c83d3cf79f23a3be983b0f7c79ed70d3ca5 not found: ID does not exist"
Sep 19 05:47:40.204260 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.204202    2581 scope.go:117] "RemoveContainer" containerID="2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43"
Sep 19 05:47:40.210535 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.210509    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.211958 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.211942    2581 scope.go:117] "RemoveContainer" containerID="2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43"
Sep 19 05:47:40.212248 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.212219    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43\": container with ID starting with 2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43 not found: ID does not exist" containerID="2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43"
Sep 19 05:47:40.212315 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.212257    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43"} err="failed to get container status \"2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43\": rpc error: code = NotFound desc = could not find container \"2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43\": container with ID starting with 2b4e4d1d67f1b0b2f1da93b775f6bf2086c56c3fe98e52a202966b47dcaf3a43 not found: ID does not exist"
Sep 19 05:47:40.213947 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.213927    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.216482 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.216461    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-66lvg\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-66lvg"
Sep 19 05:47:40.221219 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:40.221197    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gvzkq\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gvzkq"
Sep 19 05:47:40.232410 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.232392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:40.240411 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.240392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:40.476205 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.476160    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26ec8ea8-6a64-4509-8f76-2706900b8b14" path="/var/lib/kubelet/pods/26ec8ea8-6a64-4509-8f76-2706900b8b14/volumes"
Sep 19 05:47:40.476603 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.476583    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92fa6304-a769-4fe2-8e46-c08a1975e4c8" path="/var/lib/kubelet/pods/92fa6304-a769-4fe2-8e46-c08a1975e4c8/volumes"
Sep 19 05:47:40.539119 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.539096    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"]
Sep 19 05:47:40.541720 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:40.541686    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5335b629_fa3b_4277_a42e_7d8fb96d8b8f.slice/crio-2c79067208d7b46bd528813f49174c3629a5f4832029c9262bbf255eba2c518f WatchSource:0}: Error finding container 2c79067208d7b46bd528813f49174c3629a5f4832029c9262bbf255eba2c518f: Status 404 returned error can't find the container with id 2c79067208d7b46bd528813f49174c3629a5f4832029c9262bbf255eba2c518f
Sep 19 05:47:40.551341 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:40.551317    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"]
Sep 19 05:47:40.554888 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:40.554864    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9826c18_e90d_48b2_b98b_ea68840ffcc8.slice/crio-bf8eca4919d2c26e4a7f92268d07eea3e440e779a43b94b1ca50f4052f7553af WatchSource:0}: Error finding container bf8eca4919d2c26e4a7f92268d07eea3e440e779a43b94b1ca50f4052f7553af: Status 404 returned error can't find the container with id bf8eca4919d2c26e4a7f92268d07eea3e440e779a43b94b1ca50f4052f7553af
Sep 19 05:47:41.207212 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.207121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8" event={"ID":"5335b629-fa3b-4277-a42e-7d8fb96d8b8f","Type":"ContainerStarted","Data":"8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b"}
Sep 19 05:47:41.207620 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.207328    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:41.207620 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.207365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8" event={"ID":"5335b629-fa3b-4277-a42e-7d8fb96d8b8f","Type":"ContainerStarted","Data":"2c79067208d7b46bd528813f49174c3629a5f4832029c9262bbf255eba2c518f"}
Sep 19 05:47:41.208682 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.208660    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw" event={"ID":"d9826c18-e90d-48b2-b98b-ea68840ffcc8","Type":"ContainerStarted","Data":"6e282074e291d320ac387d171cd7d10e9c107af27380343b355c23e4d28a3980"}
Sep 19 05:47:41.208794 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.208700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw" event={"ID":"d9826c18-e90d-48b2-b98b-ea68840ffcc8","Type":"ContainerStarted","Data":"bf8eca4919d2c26e4a7f92268d07eea3e440e779a43b94b1ca50f4052f7553af"}
Sep 19 05:47:41.208794 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.208727    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:41.292681 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.292619    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8" podStartSLOduration=2.292599022 podStartE2EDuration="2.292599022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:41.258835055 +0000 UTC m=+406.869515039" watchObservedRunningTime="2026-09-19 05:47:41.292599022 +0000 UTC m=+406.903278990"
Sep 19 05:47:41.293385 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.293350    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw" podStartSLOduration=2.29333795 podStartE2EDuration="2.29333795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:41.291939579 +0000 UTC m=+406.902619556" watchObservedRunningTime="2026-09-19 05:47:41.29333795 +0000 UTC m=+406.904017914"
Sep 19 05:47:41.701603 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.701572    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"]
Sep 19 05:47:41.704847 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.704831    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:41.740844 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.740807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"]
Sep 19 05:47:41.797938 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.797906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d370e9-70e2-43d6-8284-416734b7bcb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dhbsj\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:41.798103 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.797947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsqrx\" (UniqueName: \"kubernetes.io/projected/f5d370e9-70e2-43d6-8284-416734b7bcb9-kube-api-access-nsqrx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dhbsj\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:41.898422 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.898392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d370e9-70e2-43d6-8284-416734b7bcb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dhbsj\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:41.898569 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.898431    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsqrx\" (UniqueName: \"kubernetes.io/projected/f5d370e9-70e2-43d6-8284-416734b7bcb9-kube-api-access-nsqrx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dhbsj\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:41.898766 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.898746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d370e9-70e2-43d6-8284-416734b7bcb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dhbsj\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:41.911898 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:41.911865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsqrx\" (UniqueName: \"kubernetes.io/projected/f5d370e9-70e2-43d6-8284-416734b7bcb9-kube-api-access-nsqrx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dhbsj\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:42.014163 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:42.014087    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:42.331901 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:42.331873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"]
Sep 19 05:47:42.333012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:47:42.332983    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d370e9_70e2_43d6_8284_416734b7bcb9.slice/crio-cdbde243e7a1d174eec88afbf391d9ac4fec9daeefa06aee6103fc4893476f1c WatchSource:0}: Error finding container cdbde243e7a1d174eec88afbf391d9ac4fec9daeefa06aee6103fc4893476f1c: Status 404 returned error can't find the container with id cdbde243e7a1d174eec88afbf391d9ac4fec9daeefa06aee6103fc4893476f1c
Sep 19 05:47:43.219488 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:43.219449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj" event={"ID":"f5d370e9-70e2-43d6-8284-416734b7bcb9","Type":"ContainerStarted","Data":"7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035"}
Sep 19 05:47:43.219630 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:43.219518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:43.219630 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:43.219536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj" event={"ID":"f5d370e9-70e2-43d6-8284-416734b7bcb9","Type":"ContainerStarted","Data":"cdbde243e7a1d174eec88afbf391d9ac4fec9daeefa06aee6103fc4893476f1c"}
Sep 19 05:47:43.264290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:43.264236    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj" podStartSLOduration=2.26422161 podStartE2EDuration="2.26422161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:43.2641239 +0000 UTC m=+408.874803874" watchObservedRunningTime="2026-09-19 05:47:43.26422161 +0000 UTC m=+408.874901574"
Sep 19 05:47:52.215670 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:52.215643    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jjtpw"
Sep 19 05:47:52.216199 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:52.215709    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:54.226734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.226704    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 05:47:54.303503 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.303470    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"]
Sep 19 05:47:54.303733 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.303683    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8" podUID="5335b629-fa3b-4277-a42e-7d8fb96d8b8f" containerName="manager" containerID="cri-o://8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b" gracePeriod=10
Sep 19 05:47:54.545887 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.545862    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:54.602864 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.602828    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26pxv\" (UniqueName: \"kubernetes.io/projected/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-kube-api-access-26pxv\") pod \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") "
Sep 19 05:47:54.603057 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.602882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-extensions-socket-volume\") pod \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\" (UID: \"5335b629-fa3b-4277-a42e-7d8fb96d8b8f\") "
Sep 19 05:47:54.603369 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.603333    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5335b629-fa3b-4277-a42e-7d8fb96d8b8f" (UID: "5335b629-fa3b-4277-a42e-7d8fb96d8b8f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:54.605107 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.605081    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-kube-api-access-26pxv" (OuterVolumeSpecName: "kube-api-access-26pxv") pod "5335b629-fa3b-4277-a42e-7d8fb96d8b8f" (UID: "5335b629-fa3b-4277-a42e-7d8fb96d8b8f"). InnerVolumeSpecName "kube-api-access-26pxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:54.703628 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.703601    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26pxv\" (UniqueName: \"kubernetes.io/projected/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-kube-api-access-26pxv\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:47:54.703628 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:54.703626    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5335b629-fa3b-4277-a42e-7d8fb96d8b8f-extensions-socket-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:47:55.273105 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.273072    2581 generic.go:356] "Generic (PLEG): container finished" podID="5335b629-fa3b-4277-a42e-7d8fb96d8b8f" containerID="8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b" exitCode=0
Sep 19 05:47:55.273497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.273126    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"
Sep 19 05:47:55.273497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.273155    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8" event={"ID":"5335b629-fa3b-4277-a42e-7d8fb96d8b8f","Type":"ContainerDied","Data":"8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b"}
Sep 19 05:47:55.273497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.273230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8" event={"ID":"5335b629-fa3b-4277-a42e-7d8fb96d8b8f","Type":"ContainerDied","Data":"2c79067208d7b46bd528813f49174c3629a5f4832029c9262bbf255eba2c518f"}
Sep 19 05:47:55.273497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.273252    2581 scope.go:117] "RemoveContainer" containerID="8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b"
Sep 19 05:47:55.282063 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.282049    2581 scope.go:117] "RemoveContainer" containerID="8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b"
Sep 19 05:47:55.282344 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:47:55.282324    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b\": container with ID starting with 8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b not found: ID does not exist" containerID="8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b"
Sep 19 05:47:55.282390 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.282352    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b"} err="failed to get container status \"8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b\": rpc error: code = NotFound desc = could not find container \"8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b\": container with ID starting with 8e23e1f86af4acbf67a36c98b6395f2580b2400a577ff284618103aabf60555b not found: ID does not exist"
Sep 19 05:47:55.297410 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.297364    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"]
Sep 19 05:47:55.300621 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:55.300602    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t9jx8"]
Sep 19 05:47:56.476620 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:47:56.476588    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5335b629-fa3b-4277-a42e-7d8fb96d8b8f" path="/var/lib/kubelet/pods/5335b629-fa3b-4277-a42e-7d8fb96d8b8f/volumes"
Sep 19 05:48:13.326957 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.326882    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:48:13.327421 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.327219    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5335b629-fa3b-4277-a42e-7d8fb96d8b8f" containerName="manager"
Sep 19 05:48:13.327421 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.327230    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5335b629-fa3b-4277-a42e-7d8fb96d8b8f" containerName="manager"
Sep 19 05:48:13.327421 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.327278    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5335b629-fa3b-4277-a42e-7d8fb96d8b8f" containerName="manager"
Sep 19 05:48:13.331654 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.331632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.334346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.334326    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 05:48:13.341713 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.341690    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:48:13.375093 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.375057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:48:13.448008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.447980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b8e91fb-1c4e-4158-a43a-dec85878969d-config-file\") pod \"limitador-limitador-84994f7ffc-clnxg\" (UID: \"9b8e91fb-1c4e-4158-a43a-dec85878969d\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.448166 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.448011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmcnm\" (UniqueName: \"kubernetes.io/projected/9b8e91fb-1c4e-4158-a43a-dec85878969d-kube-api-access-jmcnm\") pod \"limitador-limitador-84994f7ffc-clnxg\" (UID: \"9b8e91fb-1c4e-4158-a43a-dec85878969d\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.549087 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.549058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b8e91fb-1c4e-4158-a43a-dec85878969d-config-file\") pod \"limitador-limitador-84994f7ffc-clnxg\" (UID: \"9b8e91fb-1c4e-4158-a43a-dec85878969d\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.549243 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.549091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmcnm\" (UniqueName: \"kubernetes.io/projected/9b8e91fb-1c4e-4158-a43a-dec85878969d-kube-api-access-jmcnm\") pod \"limitador-limitador-84994f7ffc-clnxg\" (UID: \"9b8e91fb-1c4e-4158-a43a-dec85878969d\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.549723 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.549697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b8e91fb-1c4e-4158-a43a-dec85878969d-config-file\") pod \"limitador-limitador-84994f7ffc-clnxg\" (UID: \"9b8e91fb-1c4e-4158-a43a-dec85878969d\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.558807 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.558780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmcnm\" (UniqueName: \"kubernetes.io/projected/9b8e91fb-1c4e-4158-a43a-dec85878969d-kube-api-access-jmcnm\") pod \"limitador-limitador-84994f7ffc-clnxg\" (UID: \"9b8e91fb-1c4e-4158-a43a-dec85878969d\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.643215 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.643153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:13.928226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:13.928054    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:48:13.930803 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:48:13.930767    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8e91fb_1c4e_4158_a43a_dec85878969d.slice/crio-a028a5904aee892a2581b8cf7b13cc7043341dae8bab43c164b62959c13799fe WatchSource:0}: Error finding container a028a5904aee892a2581b8cf7b13cc7043341dae8bab43c164b62959c13799fe: Status 404 returned error can't find the container with id a028a5904aee892a2581b8cf7b13cc7043341dae8bab43c164b62959c13799fe
Sep 19 05:48:14.360336 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:14.360301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg" event={"ID":"9b8e91fb-1c4e-4158-a43a-dec85878969d","Type":"ContainerStarted","Data":"a028a5904aee892a2581b8cf7b13cc7043341dae8bab43c164b62959c13799fe"}
Sep 19 05:48:15.366499 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:15.366421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg" event={"ID":"9b8e91fb-1c4e-4158-a43a-dec85878969d","Type":"ContainerStarted","Data":"f58ea5c4c64e0c443275f41d1497cc9b61fc4b4799f03dd0ae28d56510f35fcc"}
Sep 19 05:48:15.366847 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:15.366544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:15.386144 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:15.386099    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg" podStartSLOduration=1.254892454 podStartE2EDuration="2.386085854s" totalImagesPullingTime="1.1311934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:15.384371617 +0000 UTC m=+440.995051624" watchObservedRunningTime="2026-09-19 05:48:15.386085854 +0000 UTC m=+440.996765817"
Sep 19 05:48:26.371756 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:26.371721    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-clnxg"
Sep 19 05:48:42.321216 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.321183    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c4cd99d8-ggxc4"]
Sep 19 05:48:42.324341 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.324326    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:42.327572 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.327549    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vknd7\""
Sep 19 05:48:42.332372 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.332348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c4cd99d8-ggxc4"]
Sep 19 05:48:42.388010 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.387966    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c4cd99d8-ggxc4"]
Sep 19 05:48:42.388230 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:48:42.388209    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4q6kj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-56c4cd99d8-ggxc4" podUID="23e49378-4d59-43c4-bc8a-433a0f477d7d"
Sep 19 05:48:42.468818 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.468794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q6kj\" (UniqueName: \"kubernetes.io/projected/23e49378-4d59-43c4-bc8a-433a0f477d7d-kube-api-access-4q6kj\") pod \"authorino-56c4cd99d8-ggxc4\" (UID: \"23e49378-4d59-43c4-bc8a-433a0f477d7d\") " pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:42.489893 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.489866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:42.496289 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.496270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:42.569290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.569219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q6kj\" (UniqueName: \"kubernetes.io/projected/23e49378-4d59-43c4-bc8a-433a0f477d7d-kube-api-access-4q6kj\") pod \"authorino-56c4cd99d8-ggxc4\" (UID: \"23e49378-4d59-43c4-bc8a-433a0f477d7d\") " pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:42.582800 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.582747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q6kj\" (UniqueName: \"kubernetes.io/projected/23e49378-4d59-43c4-bc8a-433a0f477d7d-kube-api-access-4q6kj\") pod \"authorino-56c4cd99d8-ggxc4\" (UID: \"23e49378-4d59-43c4-bc8a-433a0f477d7d\") " pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:42.770259 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.770229    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q6kj\" (UniqueName: \"kubernetes.io/projected/23e49378-4d59-43c4-bc8a-433a0f477d7d-kube-api-access-4q6kj\") pod \"23e49378-4d59-43c4-bc8a-433a0f477d7d\" (UID: \"23e49378-4d59-43c4-bc8a-433a0f477d7d\") "
Sep 19 05:48:42.772505 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.772479    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e49378-4d59-43c4-bc8a-433a0f477d7d-kube-api-access-4q6kj" (OuterVolumeSpecName: "kube-api-access-4q6kj") pod "23e49378-4d59-43c4-bc8a-433a0f477d7d" (UID: "23e49378-4d59-43c4-bc8a-433a0f477d7d"). InnerVolumeSpecName "kube-api-access-4q6kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:48:42.871125 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:42.871059    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q6kj\" (UniqueName: \"kubernetes.io/projected/23e49378-4d59-43c4-bc8a-433a0f477d7d-kube-api-access-4q6kj\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:48:43.494258 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:43.494232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c4cd99d8-ggxc4"
Sep 19 05:48:43.538760 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:43.538723    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c4cd99d8-ggxc4"]
Sep 19 05:48:43.545242 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:43.545218    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c4cd99d8-ggxc4"]
Sep 19 05:48:44.476737 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:48:44.476703    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23e49378-4d59-43c4-bc8a-433a0f477d7d" path="/var/lib/kubelet/pods/23e49378-4d59-43c4-bc8a-433a0f477d7d/volumes"
Sep 19 05:49:26.753627 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:49:26.753589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:49:51.647948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:49:51.647867    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:50:09.137348 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:09.137301    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:50:22.755143 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.755105    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"]
Sep 19 05:50:22.758305 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.758285    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.760798 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.760779    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 05:50:22.762000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.761982    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pg7v7\""
Sep 19 05:50:22.762089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.761982    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 19 05:50:22.762089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.762064    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 05:50:22.768695 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.768671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"]
Sep 19 05:50:22.836300 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.836253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldk29\" (UniqueName: \"kubernetes.io/projected/9c2a86e9-46b0-430e-a0ef-56513d577b7b-kube-api-access-ldk29\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.836443 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.836336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.836443 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.836361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2a86e9-46b0-430e-a0ef-56513d577b7b-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.836443 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.836392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.836443 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.836438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.836586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.836454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.937822 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.937793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldk29\" (UniqueName: \"kubernetes.io/projected/9c2a86e9-46b0-430e-a0ef-56513d577b7b-kube-api-access-ldk29\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.937989 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.937834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.937989 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.937852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2a86e9-46b0-430e-a0ef-56513d577b7b-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.937989 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.937873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.937989 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.937894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.937989 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.937908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.938297 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.938259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.938357 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.938327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.938405 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.938356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.940355 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.940338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c2a86e9-46b0-430e-a0ef-56513d577b7b-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.940634 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.940616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c2a86e9-46b0-430e-a0ef-56513d577b7b-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:22.945509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:22.945487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldk29\" (UniqueName: \"kubernetes.io/projected/9c2a86e9-46b0-430e-a0ef-56513d577b7b-kube-api-access-ldk29\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v\" (UID: \"9c2a86e9-46b0-430e-a0ef-56513d577b7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:23.069022 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:23.068994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:23.341613 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:23.341583    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"]
Sep 19 05:50:23.343988 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:50:23.343960    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2a86e9_46b0_430e_a0ef_56513d577b7b.slice/crio-6e43433d12bf437a3fd54aad10573c845b84e9e8fa61c735c39fc9c8b195d763 WatchSource:0}: Error finding container 6e43433d12bf437a3fd54aad10573c845b84e9e8fa61c735c39fc9c8b195d763: Status 404 returned error can't find the container with id 6e43433d12bf437a3fd54aad10573c845b84e9e8fa61c735c39fc9c8b195d763
Sep 19 05:50:23.932815 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:23.932784    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v" event={"ID":"9c2a86e9-46b0-430e-a0ef-56513d577b7b","Type":"ContainerStarted","Data":"6e43433d12bf437a3fd54aad10573c845b84e9e8fa61c735c39fc9c8b195d763"}
Sep 19 05:50:23.946433 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:23.946400    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:50:24.939449 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:24.939412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v" event={"ID":"9c2a86e9-46b0-430e-a0ef-56513d577b7b","Type":"ContainerStarted","Data":"bda84a768cca9a13c3646682307088401c8695d64516e6c53f66b31c1b69caf7"}
Sep 19 05:50:24.957008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:24.956965    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v" podStartSLOduration=1.991887716 podStartE2EDuration="2.956950412s" totalImagesPullingTime="965.062696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:50:24.955157811 +0000 UTC m=+570.565837787" watchObservedRunningTime="2026-09-19 05:50:24.956950412 +0000 UTC m=+570.567630375"
Sep 19 05:50:33.069423 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:33.069384    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:33.069820 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:33.069449    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:33.082156 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:33.082130    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:33.993330 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:33.993300    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v"
Sep 19 05:50:36.742820 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:36.742783    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:50:43.815154 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.815119    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"]
Sep 19 05:50:43.818555 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.818533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:43.822281 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.822251    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 19 05:50:43.848781 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.848759    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"]
Sep 19 05:50:43.909991 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.909964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfhcw\" (UniqueName: \"kubernetes.io/projected/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-kube-api-access-rfhcw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:43.910161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.910001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:43.910161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.910059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:43.910161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.910083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:43.910161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.910102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:43.910300 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:43.910165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011335 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfhcw\" (UniqueName: \"kubernetes.io/projected/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-kube-api-access-rfhcw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011335 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011746 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011888 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.011982 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.012064 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.011972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.013668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.013647    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.014096 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.014077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.019939 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.019907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfhcw\" (UniqueName: \"kubernetes.io/projected/9c8f9b06-5246-4c65-81ea-ce3e3d9091a8-kube-api-access-rfhcw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d\" (UID: \"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.127663 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.127593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:44.401834 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:44.401750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"]
Sep 19 05:50:44.408214 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:50:44.408186    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8f9b06_5246_4c65_81ea_ce3e3d9091a8.slice/crio-83ca07640ec60e8c9e82dc00ee21c0bbb21bc6f61dc0f1e954822eda0c5d17b3 WatchSource:0}: Error finding container 83ca07640ec60e8c9e82dc00ee21c0bbb21bc6f61dc0f1e954822eda0c5d17b3: Status 404 returned error can't find the container with id 83ca07640ec60e8c9e82dc00ee21c0bbb21bc6f61dc0f1e954822eda0c5d17b3
Sep 19 05:50:45.033610 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:45.033570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d" event={"ID":"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8","Type":"ContainerStarted","Data":"5d68d62638e7abb42fa31b92583c642a29a29552b5e2114af3a572adf6a78d6a"}
Sep 19 05:50:45.033956 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:45.033638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d" event={"ID":"9c8f9b06-5246-4c65-81ea-ce3e3d9091a8","Type":"ContainerStarted","Data":"83ca07640ec60e8c9e82dc00ee21c0bbb21bc6f61dc0f1e954822eda0c5d17b3"}
Sep 19 05:50:45.050925 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:45.050884    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d" podStartSLOduration=1.889796618 podStartE2EDuration="2.050869296s" totalImagesPullingTime="161.072678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:50:45.049475576 +0000 UTC m=+590.660155567" watchObservedRunningTime="2026-09-19 05:50:45.050869296 +0000 UTC m=+590.661549260"
Sep 19 05:50:45.545736 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:45.545706    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:50:52.946610 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:52.946577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:50:54.128532 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:54.128502    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:54.128897 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:54.128566    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:54.141332 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:54.141302    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:55.086964 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:55.086935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d"
Sep 19 05:50:56.399885 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:56.399861    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:50:56.400369 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:56.399863    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:50:56.405805 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:56.405787    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:50:56.405931 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:50:56.405843    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:51:02.963833 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:02.963800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"]
Sep 19 05:51:02.967590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:02.967565    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:02.970606 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:02.970587    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 19 05:51:02.976969 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:02.976947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"]
Sep 19 05:51:03.070360 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.070332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.070533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.070366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.070533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.070384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wsfj\" (UniqueName: \"kubernetes.io/projected/ff877d56-3140-4233-bd2a-79f82f7b8f0e-kube-api-access-7wsfj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.070533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.070448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.070533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.070479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.070533 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.070502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff877d56-3140-4233-bd2a-79f82f7b8f0e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171143 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171329 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171329 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wsfj\" (UniqueName: \"kubernetes.io/projected/ff877d56-3140-4233-bd2a-79f82f7b8f0e-kube-api-access-7wsfj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171329 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171329 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171329 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff877d56-3140-4233-bd2a-79f82f7b8f0e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171597 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171597 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.171684 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.171649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.173605 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.173577    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff877d56-3140-4233-bd2a-79f82f7b8f0e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.173880 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.173866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff877d56-3140-4233-bd2a-79f82f7b8f0e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.194400 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.194375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wsfj\" (UniqueName: \"kubernetes.io/projected/ff877d56-3140-4233-bd2a-79f82f7b8f0e-kube-api-access-7wsfj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t58z2\" (UID: \"ff877d56-3140-4233-bd2a-79f82f7b8f0e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.276895 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.276864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:03.552572 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:03.552526    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"]
Sep 19 05:51:03.554703 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:51:03.554682    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff877d56_3140_4233_bd2a_79f82f7b8f0e.slice/crio-4f69b39111d4c7250a7f881f8c60e5df92a77158caf7fb518a1aba9bb84e1685 WatchSource:0}: Error finding container 4f69b39111d4c7250a7f881f8c60e5df92a77158caf7fb518a1aba9bb84e1685: Status 404 returned error can't find the container with id 4f69b39111d4c7250a7f881f8c60e5df92a77158caf7fb518a1aba9bb84e1685
Sep 19 05:51:04.118323 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:04.118283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2" event={"ID":"ff877d56-3140-4233-bd2a-79f82f7b8f0e","Type":"ContainerStarted","Data":"439b14d9ce6c8ef63d00bc75c990531548af35d292d3a8b090f83794af4a2b2e"}
Sep 19 05:51:04.118323 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:04.118323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2" event={"ID":"ff877d56-3140-4233-bd2a-79f82f7b8f0e","Type":"ContainerStarted","Data":"4f69b39111d4c7250a7f881f8c60e5df92a77158caf7fb518a1aba9bb84e1685"}
Sep 19 05:51:04.136712 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:04.136659    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2" podStartSLOduration=1.979338266 podStartE2EDuration="2.136643859s" totalImagesPullingTime="157.305593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:51:04.134595467 +0000 UTC m=+609.745275481" watchObservedRunningTime="2026-09-19 05:51:04.136643859 +0000 UTC m=+609.747323821"
Sep 19 05:51:05.448730 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:05.448696    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:51:13.277292 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:13.277206    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:13.277658 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:13.277292    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:13.290514 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:13.290497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:51:14.175338 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:51:14.175307    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t58z2"
Sep 19 05:53:26.753166 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.753131    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8111-233009"]
Sep 19 05:53:26.756406 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.756384    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:26.760460 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.760439    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpl82\""
Sep 19 05:53:26.761268 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.761248    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8111-233009"]
Sep 19 05:53:26.855327 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.855291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm65l\" (UniqueName: \"kubernetes.io/projected/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6-kube-api-access-dm65l\") pod \"test-curl-8111-233009\" (UID: \"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6\") " pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:26.956348 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.956314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm65l\" (UniqueName: \"kubernetes.io/projected/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6-kube-api-access-dm65l\") pod \"test-curl-8111-233009\" (UID: \"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6\") " pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:26.966084 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:26.966027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm65l\" (UniqueName: \"kubernetes.io/projected/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6-kube-api-access-dm65l\") pod \"test-curl-8111-233009\" (UID: \"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6\") " pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:27.066296 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:27.066266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:27.334948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:27.334924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8111-233009"]
Sep 19 05:53:27.337319 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:27.337289    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ed4d18d_cf74_4486_99dd_fdf6cf5b3da6.slice/crio-b86296b77d7120f0ac7ec3ced1c45f5c739e3e31b51d65ea8a00ff8517a247d3 WatchSource:0}: Error finding container b86296b77d7120f0ac7ec3ced1c45f5c739e3e31b51d65ea8a00ff8517a247d3: Status 404 returned error can't find the container with id b86296b77d7120f0ac7ec3ced1c45f5c739e3e31b51d65ea8a00ff8517a247d3
Sep 19 05:53:27.339418 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:27.339400    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:53:27.761022 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:27.760946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8111-233009" event={"ID":"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6","Type":"ContainerStarted","Data":"b86296b77d7120f0ac7ec3ced1c45f5c739e3e31b51d65ea8a00ff8517a247d3"}
Sep 19 05:53:29.446160 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:29.446123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:53:29.772793 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:29.772715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8111-233009" event={"ID":"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6","Type":"ContainerStarted","Data":"de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66"}
Sep 19 05:53:29.794494 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:29.794433    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8111-233009" podStartSLOduration=1.570387131 podStartE2EDuration="3.794412185s" totalImagesPullingTime="2.224025054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:29.792802433 +0000 UTC m=+755.403482397" watchObservedRunningTime="2026-09-19 05:53:29.794412185 +0000 UTC m=+755.405092151"
Sep 19 05:53:30.440047 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.440000    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8111-233009"]
Sep 19 05:53:30.444195 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.444170    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8111-233009"]
Sep 19 05:53:30.568094 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.568065    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8111-8e9109"]
Sep 19 05:53:30.568439 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.568364    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" containerName="test-curl-8111-233009"
Sep 19 05:53:30.568439 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.568375    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" containerName="test-curl-8111-233009"
Sep 19 05:53:30.568509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.568443    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" containerName="test-curl-8111-233009"
Sep 19 05:53:30.571487 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.571472    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:30.574883 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.574858    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8111-8e9109"]
Sep 19 05:53:30.687808 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.687768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk58g\" (UniqueName: \"kubernetes.io/projected/0d8c8f92-c866-4296-bd90-c92cdc53aa08-kube-api-access-xk58g\") pod \"test-curl-8111-8e9109\" (UID: \"0d8c8f92-c866-4296-bd90-c92cdc53aa08\") " pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:30.777981 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:30.777947    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8111-233009" container="test-curl-8111-233009"
Sep 19 05:53:30.789199 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.789167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk58g\" (UniqueName: \"kubernetes.io/projected/0d8c8f92-c866-4296-bd90-c92cdc53aa08-kube-api-access-xk58g\") pod \"test-curl-8111-8e9109\" (UID: \"0d8c8f92-c866-4296-bd90-c92cdc53aa08\") " pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:30.797935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.797908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk58g\" (UniqueName: \"kubernetes.io/projected/0d8c8f92-c866-4296-bd90-c92cdc53aa08-kube-api-access-xk58g\") pod \"test-curl-8111-8e9109\" (UID: \"0d8c8f92-c866-4296-bd90-c92cdc53aa08\") " pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:30.881163 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:30.881092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:31.154181 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:31.154108    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8111-8e9109"]
Sep 19 05:53:31.156609 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:31.156583    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d8c8f92_c866_4296_bd90_c92cdc53aa08.slice/crio-58c8eca5e0bd6a343b022492fb579ab6be1380aa6bf35e4a90ec4223a2cb1bd6 WatchSource:0}: Error finding container 58c8eca5e0bd6a343b022492fb579ab6be1380aa6bf35e4a90ec4223a2cb1bd6: Status 404 returned error can't find the container with id 58c8eca5e0bd6a343b022492fb579ab6be1380aa6bf35e4a90ec4223a2cb1bd6
Sep 19 05:53:31.784237 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:31.784198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8111-8e9109" event={"ID":"0d8c8f92-c866-4296-bd90-c92cdc53aa08","Type":"ContainerStarted","Data":"d552127afe22eee478cbaa2c60d0f5879160cd3a252b40fd9bbfb0eb96e16c6e"}
Sep 19 05:53:31.784237 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:31.784243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8111-8e9109" event={"ID":"0d8c8f92-c866-4296-bd90-c92cdc53aa08","Type":"ContainerStarted","Data":"58c8eca5e0bd6a343b022492fb579ab6be1380aa6bf35e4a90ec4223a2cb1bd6"}
Sep 19 05:53:31.784737 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:31.784288    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8111-233009" podUID="9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" containerName="test-curl-8111-233009" containerID="cri-o://de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66" gracePeriod=2
Sep 19 05:53:31.798855 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:31.798812    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8111-8e9109" podStartSLOduration=1.79879787 podStartE2EDuration="1.79879787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:31.798243725 +0000 UTC m=+757.408923692" watchObservedRunningTime="2026-09-19 05:53:31.79879787 +0000 UTC m=+757.409477828"
Sep 19 05:53:32.425433 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.425400    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8111-8e9109"]
Sep 19 05:53:32.427488 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.427462    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8111-8e9109"]
Sep 19 05:53:32.566558 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.566527    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"]
Sep 19 05:53:32.566906 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.566890    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" containerName="test-curl-8111-8e9109"
Sep 19 05:53:32.566906 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.566906    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" containerName="test-curl-8111-8e9109"
Sep 19 05:53:32.567060 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.566978    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" containerName="test-curl-8111-8e9109"
Sep 19 05:53:32.570255 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.570236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.574000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.573969    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-cc0e26-serving-cert\""
Sep 19 05:53:32.574182 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.573999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nkngb\""
Sep 19 05:53:32.574735 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.574643    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 05:53:32.574735 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.574699    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 05:53:32.574942 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.574649    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 05:53:32.580001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.579979    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"]
Sep 19 05:53:32.706880 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.706847    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zxsq\" (UniqueName: \"kubernetes.io/projected/f1f8efde-3b9a-4119-b0e7-726792243e47-kube-api-access-4zxsq\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.707084 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.706898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.707084 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.706938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.707084 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.706971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f8efde-3b9a-4119-b0e7-726792243e47-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.789065 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:32.789003    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8111-8e9109" container="test-curl-8111-8e9109"
Sep 19 05:53:32.808220 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.808194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zxsq\" (UniqueName: \"kubernetes.io/projected/f1f8efde-3b9a-4119-b0e7-726792243e47-kube-api-access-4zxsq\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.808356 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.808230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.808356 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.808254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.808356 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.808274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f8efde-3b9a-4119-b0e7-726792243e47-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.808979 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.808956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f8efde-3b9a-4119-b0e7-726792243e47-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.810768 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.810747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.810873 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.810774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.819347 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.819301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zxsq\" (UniqueName: \"kubernetes.io/projected/f1f8efde-3b9a-4119-b0e7-726792243e47-kube-api-access-4zxsq\") pod \"maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:32.882315 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:32.882289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:33.156148 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.156020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"]
Sep 19 05:53:33.162884 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:33.162852    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f8efde_3b9a_4119_b0e7_726792243e47.slice/crio-d01016ff9c2e0cc866ad56b5eaba402188d5f21b63d8a6ac24b6998252716746 WatchSource:0}: Error finding container d01016ff9c2e0cc866ad56b5eaba402188d5f21b63d8a6ac24b6998252716746: Status 404 returned error can't find the container with id d01016ff9c2e0cc866ad56b5eaba402188d5f21b63d8a6ac24b6998252716746
Sep 19 05:53:33.309094 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.309058    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"]
Sep 19 05:53:33.314224 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.314201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.317857 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.317760    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-8128f3-openshift-default-dockercfg-rkrd4\""
Sep 19 05:53:33.318256 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.318236    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 05:53:33.330241 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.330218    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"]
Sep 19 05:53:33.414223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-socket\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istiod-ca-cert\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-data\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w48f9\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-kube-api-access-w48f9\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414445 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-credential-socket\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414445 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-envoy\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414445 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-token\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414445 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-certs\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.414607 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.414448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-podinfo\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.515903 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.515863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-credential-socket\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.515957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-envoy\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.515981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-token\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516067 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-certs\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516099    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-podinfo\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-socket\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516248 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istiod-ca-cert\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516248 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-data\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516334 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-credential-socket\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516385 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-certs\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516385 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w48f9\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-kube-api-access-w48f9\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516504 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-socket\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516786 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-data\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.516786 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.516833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istiod-ca-cert\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.518461 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.518440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-envoy\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.518594 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.518578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-podinfo\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.530571 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.530548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-token\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.530679 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.530663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w48f9\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-kube-api-access-w48f9\") pod \"e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") " pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.626384 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.626347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:33.803008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.802648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk" event={"ID":"f1f8efde-3b9a-4119-b0e7-726792243e47","Type":"ContainerStarted","Data":"d01016ff9c2e0cc866ad56b5eaba402188d5f21b63d8a6ac24b6998252716746"}
Sep 19 05:53:33.803008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.802699    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8111-8e9109" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" containerName="test-curl-8111-8e9109" containerID="cri-o://d552127afe22eee478cbaa2c60d0f5879160cd3a252b40fd9bbfb0eb96e16c6e" gracePeriod=2
Sep 19 05:53:33.927986 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.919254    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"]
Sep 19 05:53:33.927986 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:33.922521    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55d0bcf_5ee8_4f99_a55b_736b1da5bbc1.slice/crio-83a10e6d4d906669699beec1dc99e4356c449afb3c67bffcb824b408cef87679 WatchSource:0}: Error finding container 83a10e6d4d906669699beec1dc99e4356c449afb3c67bffcb824b408cef87679: Status 404 returned error can't find the container with id 83a10e6d4d906669699beec1dc99e4356c449afb3c67bffcb824b408cef87679
Sep 19 05:53:33.972683 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:33.972655    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:34.123248 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.123198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm65l\" (UniqueName: \"kubernetes.io/projected/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6-kube-api-access-dm65l\") pod \"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6\" (UID: \"9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6\") "
Sep 19 05:53:34.125878 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.125841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6-kube-api-access-dm65l" (OuterVolumeSpecName: "kube-api-access-dm65l") pod "9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" (UID: "9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6"). InnerVolumeSpecName "kube-api-access-dm65l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:53:34.224589 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.224504    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm65l\" (UniqueName: \"kubernetes.io/projected/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6-kube-api-access-dm65l\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:53:34.485505 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.485390    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" path="/var/lib/kubelet/pods/9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6/volumes"
Sep 19 05:53:34.810234 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.810195    2581 generic.go:356] "Generic (PLEG): container finished" podID="9ed4d18d-cf74-4486-99dd-fdf6cf5b3da6" containerID="de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66" exitCode=137
Sep 19 05:53:34.810695 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.810254    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-233009"
Sep 19 05:53:34.810695 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.810326    2581 scope.go:117] "RemoveContainer" containerID="de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66"
Sep 19 05:53:34.813331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:34.813305    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" event={"ID":"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1","Type":"ContainerStarted","Data":"83a10e6d4d906669699beec1dc99e4356c449afb3c67bffcb824b408cef87679"}
Sep 19 05:53:35.660877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:35.660858    2581 scope.go:117] "RemoveContainer" containerID="de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66"
Sep 19 05:53:35.661330 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:35.661307    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66\": container with ID starting with de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66 not found: ID does not exist" containerID="de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66"
Sep 19 05:53:35.661409 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:35.661345    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66"} err="failed to get container status \"de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66\": rpc error: code = NotFound desc = could not find container \"de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66\": container with ID starting with de9f9f517d904720a30a16f700fb319788c4c403b714710409cba2d65cd7cc66 not found: ID does not exist"
Sep 19 05:53:36.737362 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.737328    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:53:36.737679 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.737400    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:53:36.737679 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.737430    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:53:36.831601 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.831574    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" containerID="d552127afe22eee478cbaa2c60d0f5879160cd3a252b40fd9bbfb0eb96e16c6e" exitCode=137
Sep 19 05:53:36.854181 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.854127    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podStartSLOduration=1.043831184 podStartE2EDuration="3.854108419s" totalImagesPullingTime="2.810277235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:36.853598322 +0000 UTC m=+762.464278284" watchObservedRunningTime="2026-09-19 05:53:36.854108419 +0000 UTC m=+762.464788384"
Sep 19 05:53:36.875918 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.875864    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:36.878426 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:36.878370    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8111-8e9109\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:36.953598 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.953518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk58g\" (UniqueName: \"kubernetes.io/projected/0d8c8f92-c866-4296-bd90-c92cdc53aa08-kube-api-access-xk58g\") pod \"0d8c8f92-c866-4296-bd90-c92cdc53aa08\" (UID: \"0d8c8f92-c866-4296-bd90-c92cdc53aa08\") "
Sep 19 05:53:36.955643 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:36.955616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d8c8f92-c866-4296-bd90-c92cdc53aa08-kube-api-access-xk58g" (OuterVolumeSpecName: "kube-api-access-xk58g") pod "0d8c8f92-c866-4296-bd90-c92cdc53aa08" (UID: "0d8c8f92-c866-4296-bd90-c92cdc53aa08"). InnerVolumeSpecName "kube-api-access-xk58g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:53:37.055011 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.054975    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk58g\" (UniqueName: \"kubernetes.io/projected/0d8c8f92-c866-4296-bd90-c92cdc53aa08-kube-api-access-xk58g\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:53:37.626735 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.626694    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:37.631798 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.631771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:37.655700 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:37.655662    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8111-8e9109\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:37.837632 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.837596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk" event={"ID":"f1f8efde-3b9a-4119-b0e7-726792243e47","Type":"ContainerStarted","Data":"4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab"}
Sep 19 05:53:37.838110 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.837703    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:37.839385 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.839359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" event={"ID":"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1","Type":"ContainerStarted","Data":"85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"}
Sep 19 05:53:37.839546 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.839531    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:37.840462 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.840444    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:53:37.840667 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.840653    2581 scope.go:117] "RemoveContainer" containerID="d552127afe22eee478cbaa2c60d0f5879160cd3a252b40fd9bbfb0eb96e16c6e"
Sep 19 05:53:37.840717 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.840666    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:37.855991 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:37.855952    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk" podStartSLOduration=2.286452198 podStartE2EDuration="5.855938719s" totalImagesPullingTime="3.569486521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:37.854514291 +0000 UTC m=+763.465194276" watchObservedRunningTime="2026-09-19 05:53:37.855938719 +0000 UTC m=+763.466618683"
Sep 19 05:53:37.856872 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:37.856848    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8111-8e9109\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:37.882123 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:37.882062    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8111-8e9109\" is forbidden: User \"system:node:ip-10-0-131-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-202.ec2.internal' and this object" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" pod="opendatahub/test-curl-8111-8e9109"
Sep 19 05:53:38.478134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:38.478100    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d8c8f92-c866-4296-bd90-c92cdc53aa08" path="/var/lib/kubelet/pods/0d8c8f92-c866-4296-bd90-c92cdc53aa08/volumes"
Sep 19 05:53:39.364616 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.364581    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"]
Sep 19 05:53:39.369251 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.369235    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.371950 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.371926    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bd4c1118-dockercfg-dx26n\""
Sep 19 05:53:39.372074 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.371929    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bd4c1118\""
Sep 19 05:53:39.379059 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.379011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"]
Sep 19 05:53:39.477104 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.477061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2wn6\" (UniqueName: \"kubernetes.io/projected/a0de6f50-cdd9-4fe9-960f-140c8839895e-kube-api-access-d2wn6\") pod \"payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.477248 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.477140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0de6f50-cdd9-4fe9-960f-140c8839895e-plugins-config-volume\") pod \"payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.578575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.578544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2wn6\" (UniqueName: \"kubernetes.io/projected/a0de6f50-cdd9-4fe9-960f-140c8839895e-kube-api-access-d2wn6\") pod \"payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.578575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.578576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0de6f50-cdd9-4fe9-960f-140c8839895e-plugins-config-volume\") pod \"payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.579248 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.579229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0de6f50-cdd9-4fe9-960f-140c8839895e-plugins-config-volume\") pod \"payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.587825 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.587796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2wn6\" (UniqueName: \"kubernetes.io/projected/a0de6f50-cdd9-4fe9-960f-140c8839895e-kube-api-access-d2wn6\") pod \"payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.680698 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.680632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:39.959896 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:39.959835    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"]
Sep 19 05:53:39.962071 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:39.962010    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0de6f50_cdd9_4fe9_960f_140c8839895e.slice/crio-a01887bf6fe8ea2fce44332214d85cacddfc60f59e3966b65c9d53376a719f0d WatchSource:0}: Error finding container a01887bf6fe8ea2fce44332214d85cacddfc60f59e3966b65c9d53376a719f0d: Status 404 returned error can't find the container with id a01887bf6fe8ea2fce44332214d85cacddfc60f59e3966b65c9d53376a719f0d
Sep 19 05:53:40.860209 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:40.860170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf" event={"ID":"a0de6f50-cdd9-4fe9-960f-140c8839895e","Type":"ContainerStarted","Data":"a01887bf6fe8ea2fce44332214d85cacddfc60f59e3966b65c9d53376a719f0d"}
Sep 19 05:53:42.617318 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.617134    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"]
Sep 19 05:53:42.620704 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.620679    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.624883 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.624864    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-8128f3-serving-cert\""
Sep 19 05:53:42.636582 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.636561    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"]
Sep 19 05:53:42.708381 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.708344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-openshift-service-ca\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.708381 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.708382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-metrics-certs\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.708614 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.708459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.708614 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.708484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksvnt\" (UniqueName: \"kubernetes.io/projected/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-kube-api-access-ksvnt\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.809403 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.809333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.809403 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.809378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksvnt\" (UniqueName: \"kubernetes.io/projected/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-kube-api-access-ksvnt\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.809611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.809451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-openshift-service-ca\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.809611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.809480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-metrics-certs\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.809611 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:42.809480    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-serving-cert: secret "maas-api-e2e-autores-8128f3-serving-cert" not found
Sep 19 05:53:42.809754 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:53:42.809629    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls podName:6b4ebd4e-4648-4636-bc98-6aa7b44fab58 nodeName:}" failed. No retries permitted until 2026-09-19 05:53:43.309608821 +0000 UTC m=+768.920288779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls") pod "maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" (UID: "6b4ebd4e-4648-4636-bc98-6aa7b44fab58") : secret "maas-api-e2e-autores-8128f3-serving-cert" not found
Sep 19 05:53:42.810203 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.810180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-openshift-service-ca\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.812114 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.812092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-metrics-certs\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.821785 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.821763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksvnt\" (UniqueName: \"kubernetes.io/projected/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-kube-api-access-ksvnt\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:42.871669 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.871637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf" event={"ID":"a0de6f50-cdd9-4fe9-960f-140c8839895e","Type":"ContainerStarted","Data":"c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674"}
Sep 19 05:53:42.871785 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.871759    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:42.904285 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:42.904237    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf" podStartSLOduration=1.335437685 podStartE2EDuration="3.904223051s" totalImagesPullingTime="2.568785366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:42.902945646 +0000 UTC m=+768.513625635" watchObservedRunningTime="2026-09-19 05:53:42.904223051 +0000 UTC m=+768.514903015"
Sep 19 05:53:43.314565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.314534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:43.317203 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.317182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls\") pod \"maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:43.533134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.533100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:43.749864 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.749832    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"]
Sep 19 05:53:43.754233 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.754217    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:43.756984 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.756956    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8128f3\""
Sep 19 05:53:43.757133 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.757117    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-8128f3-dockercfg-slwzp\""
Sep 19 05:53:43.763311 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.763287    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"]
Sep 19 05:53:43.819899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.819876    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"]
Sep 19 05:53:43.821638 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:43.821609    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4ebd4e_4648_4636_bc98_6aa7b44fab58.slice/crio-2fa88e8c52a6846ca137c9d1b317c338d0ec1961bebe8445aa6343776f18f49f WatchSource:0}: Error finding container 2fa88e8c52a6846ca137c9d1b317c338d0ec1961bebe8445aa6343776f18f49f: Status 404 returned error can't find the container with id 2fa88e8c52a6846ca137c9d1b317c338d0ec1961bebe8445aa6343776f18f49f
Sep 19 05:53:43.852982 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.852922    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 05:53:43.878108 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.878070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" event={"ID":"6b4ebd4e-4648-4636-bc98-6aa7b44fab58","Type":"ContainerStarted","Data":"2fa88e8c52a6846ca137c9d1b317c338d0ec1961bebe8445aa6343776f18f49f"}
Sep 19 05:53:43.919788 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.919750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnrj6\" (UniqueName: \"kubernetes.io/projected/66c82d38-f61e-4d88-a375-ac037694c6cb-kube-api-access-pnrj6\") pod \"payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:43.919921 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:43.919831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66c82d38-f61e-4d88-a375-ac037694c6cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.021264 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.021226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnrj6\" (UniqueName: \"kubernetes.io/projected/66c82d38-f61e-4d88-a375-ac037694c6cb-kube-api-access-pnrj6\") pod \"payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.021910 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.021877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66c82d38-f61e-4d88-a375-ac037694c6cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.022510 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.022490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66c82d38-f61e-4d88-a375-ac037694c6cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.030733 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.030705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnrj6\" (UniqueName: \"kubernetes.io/projected/66c82d38-f61e-4d88-a375-ac037694c6cb-kube-api-access-pnrj6\") pod \"payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.065556 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.065531    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.339943 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.339919    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"]
Sep 19 05:53:44.342922 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:44.342901    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c82d38_f61e_4d88_a375_ac037694c6cb.slice/crio-d5bad36a0130034272d41ff0706c58e01baa6362666cd3644e4669506e35ecca WatchSource:0}: Error finding container d5bad36a0130034272d41ff0706c58e01baa6362666cd3644e4669506e35ecca: Status 404 returned error can't find the container with id d5bad36a0130034272d41ff0706c58e01baa6362666cd3644e4669506e35ecca
Sep 19 05:53:44.883951 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.883918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf" event={"ID":"66c82d38-f61e-4d88-a375-ac037694c6cb","Type":"ContainerStarted","Data":"6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247"}
Sep 19 05:53:44.884343 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.883979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf" event={"ID":"66c82d38-f61e-4d88-a375-ac037694c6cb","Type":"ContainerStarted","Data":"d5bad36a0130034272d41ff0706c58e01baa6362666cd3644e4669506e35ecca"}
Sep 19 05:53:44.884343 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.884102    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:44.903769 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:44.903721    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf" podStartSLOduration=1.9037064190000001 podStartE2EDuration="1.903706419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:44.901840212 +0000 UTC m=+770.512520200" watchObservedRunningTime="2026-09-19 05:53:44.903706419 +0000 UTC m=+770.514386382"
Sep 19 05:53:45.890543 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:45.890507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" event={"ID":"6b4ebd4e-4648-4636-bc98-6aa7b44fab58","Type":"ContainerStarted","Data":"d7bfd2883e6ce7141bb11371a4e23c9f57ecfad03d6efe0df7a417c4cb1df201"}
Sep 19 05:53:45.907842 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:45.907797    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" podStartSLOduration=2.645237994 podStartE2EDuration="3.907783301s" totalImagesPullingTime="1.262545307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:45.905953322 +0000 UTC m=+771.516633304" watchObservedRunningTime="2026-09-19 05:53:45.907783301 +0000 UTC m=+771.518463264"
Sep 19 05:53:46.896361 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:46.896325    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:49.155511 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:49.155478    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:53:52.748978 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.748941    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"]
Sep 19 05:53:52.752697 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.752681    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:52.756831 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.756800    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c85316\""
Sep 19 05:53:52.756949 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.756878    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c85316-dockercfg-2r97f\""
Sep 19 05:53:52.772933 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.772902    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"]
Sep 19 05:53:52.919190 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.919164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:53:52.922378 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.922349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/554e22bd-2655-4605-b41e-192d357145a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:52.922471 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:52.922404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27rs\" (UniqueName: \"kubernetes.io/projected/554e22bd-2655-4605-b41e-192d357145a4-kube-api-access-w27rs\") pod \"payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.023676 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.023598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/554e22bd-2655-4605-b41e-192d357145a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.023676 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.023634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27rs\" (UniqueName: \"kubernetes.io/projected/554e22bd-2655-4605-b41e-192d357145a4-kube-api-access-w27rs\") pod \"payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.024601 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.024581    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/554e22bd-2655-4605-b41e-192d357145a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.032898 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.032873    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27rs\" (UniqueName: \"kubernetes.io/projected/554e22bd-2655-4605-b41e-192d357145a4-kube-api-access-w27rs\") pod \"payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.062740 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.062715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.373465 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.373444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"]
Sep 19 05:53:53.375603 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:53:53.375576    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554e22bd_2655_4605_b41e_192d357145a4.slice/crio-f489d4ba25f4dd730ff0cb1b0d5cfe85e771cc65b22d2d13ec1d3906310b74d6 WatchSource:0}: Error finding container f489d4ba25f4dd730ff0cb1b0d5cfe85e771cc65b22d2d13ec1d3906310b74d6: Status 404 returned error can't find the container with id f489d4ba25f4dd730ff0cb1b0d5cfe85e771cc65b22d2d13ec1d3906310b74d6
Sep 19 05:53:53.880633 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.880603    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:53:53.934936 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.934905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w" event={"ID":"554e22bd-2655-4605-b41e-192d357145a4","Type":"ContainerStarted","Data":"bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d"}
Sep 19 05:53:53.935126 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.934954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w" event={"ID":"554e22bd-2655-4605-b41e-192d357145a4","Type":"ContainerStarted","Data":"f489d4ba25f4dd730ff0cb1b0d5cfe85e771cc65b22d2d13ec1d3906310b74d6"}
Sep 19 05:53:53.935126 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.935020    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:53:53.959573 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:53.959528    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w" podStartSLOduration=1.959515196 podStartE2EDuration="1.959515196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:53.958860017 +0000 UTC m=+779.569540010" watchObservedRunningTime="2026-09-19 05:53:53.959515196 +0000 UTC m=+779.570195159"
Sep 19 05:53:55.894290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:55.894242    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:53:59.967145 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:59.967073    2581 generic.go:356] "Generic (PLEG): container finished" podID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerID="85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a" exitCode=0
Sep 19 05:53:59.967509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:59.967150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" event={"ID":"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1","Type":"ContainerDied","Data":"85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"}
Sep 19 05:53:59.967549 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:59.967515    2581 scope.go:117] "RemoveContainer" containerID="85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"
Sep 19 05:53:59.968612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:59.968587    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:53:59.968665 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:59.968652    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:53:59.968699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:53:59.968681    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:00.974276 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:00.974241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" event={"ID":"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1","Type":"ContainerStarted","Data":"2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077"}
Sep 19 05:54:01.627010 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:01.626971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:54:01.627244 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:01.627224    2581 patch_prober.go:28] interesting pod/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 19 05:54:01.627318 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:01.627274    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 19 05:54:02.626927 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:02.626883    2581 patch_prober.go:28] interesting pod/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 19 05:54:02.627338 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:02.626947    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 19 05:54:03.626838 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:03.626797    2581 patch_prober.go:28] interesting pod/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 19 05:54:03.627006 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:03.626870    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 19 05:54:04.627384 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:04.627344    2581 patch_prober.go:28] interesting pod/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 19 05:54:04.627759 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:04.627402    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 19 05:54:04.942565 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:04.942489    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 05:54:05.626923 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:05.626882    2581 patch_prober.go:28] interesting pod/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 19 05:54:05.627112 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:05.626964    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 19 05:54:06.631733 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:06.631663    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:54:06.632203 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:06.632071    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:54:06.632776 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:06.632756    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:54:08.272008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.271978    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"]
Sep 19 05:54:08.272487 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.272209    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf" podUID="a0de6f50-cdd9-4fe9-960f-140c8839895e" containerName="payload-processing" containerID="cri-o://c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674" gracePeriod=30
Sep 19 05:54:08.533813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.533789    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:54:08.568224 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.568195    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2wn6\" (UniqueName: \"kubernetes.io/projected/a0de6f50-cdd9-4fe9-960f-140c8839895e-kube-api-access-d2wn6\") pod \"a0de6f50-cdd9-4fe9-960f-140c8839895e\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") "
Sep 19 05:54:08.568224 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.568225    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0de6f50-cdd9-4fe9-960f-140c8839895e-plugins-config-volume\") pod \"a0de6f50-cdd9-4fe9-960f-140c8839895e\" (UID: \"a0de6f50-cdd9-4fe9-960f-140c8839895e\") "
Sep 19 05:54:08.568750 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.568714    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0de6f50-cdd9-4fe9-960f-140c8839895e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0de6f50-cdd9-4fe9-960f-140c8839895e" (UID: "a0de6f50-cdd9-4fe9-960f-140c8839895e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:54:08.570840 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.570811    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0de6f50-cdd9-4fe9-960f-140c8839895e-kube-api-access-d2wn6" (OuterVolumeSpecName: "kube-api-access-d2wn6") pod "a0de6f50-cdd9-4fe9-960f-140c8839895e" (UID: "a0de6f50-cdd9-4fe9-960f-140c8839895e"). InnerVolumeSpecName "kube-api-access-d2wn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:54:08.669007 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.668976    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2wn6\" (UniqueName: \"kubernetes.io/projected/a0de6f50-cdd9-4fe9-960f-140c8839895e-kube-api-access-d2wn6\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:08.669007 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:08.669002    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0de6f50-cdd9-4fe9-960f-140c8839895e-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:09.020908 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.020829    2581 generic.go:356] "Generic (PLEG): container finished" podID="a0de6f50-cdd9-4fe9-960f-140c8839895e" containerID="c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674" exitCode=0
Sep 19 05:54:09.020908 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.020873    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"
Sep 19 05:54:09.021114 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.020909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf" event={"ID":"a0de6f50-cdd9-4fe9-960f-140c8839895e","Type":"ContainerDied","Data":"c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674"}
Sep 19 05:54:09.021114 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.020964    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf" event={"ID":"a0de6f50-cdd9-4fe9-960f-140c8839895e","Type":"ContainerDied","Data":"a01887bf6fe8ea2fce44332214d85cacddfc60f59e3966b65c9d53376a719f0d"}
Sep 19 05:54:09.021114 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.020987    2581 scope.go:117] "RemoveContainer" containerID="c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674"
Sep 19 05:54:09.030213 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.030195    2581 scope.go:117] "RemoveContainer" containerID="c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674"
Sep 19 05:54:09.030445 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:54:09.030426    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674\": container with ID starting with c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674 not found: ID does not exist" containerID="c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674"
Sep 19 05:54:09.030492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.030454    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674"} err="failed to get container status \"c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674\": rpc error: code = NotFound desc = could not find container \"c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674\": container with ID starting with c16f3e67c3cd0bba3589ceec7625f16193f3090cb90f2ad9f042f8734cf99674 not found: ID does not exist"
Sep 19 05:54:09.061612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.061586    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"]
Sep 19 05:54:09.068663 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.068640    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4c1118-ccb7895dc-h69gf"]
Sep 19 05:54:09.627969 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.627931    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"]
Sep 19 05:54:09.628340 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.628304    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0de6f50-cdd9-4fe9-960f-140c8839895e" containerName="payload-processing"
Sep 19 05:54:09.628340 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.628316    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0de6f50-cdd9-4fe9-960f-140c8839895e" containerName="payload-processing"
Sep 19 05:54:09.628416 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.628390    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0de6f50-cdd9-4fe9-960f-140c8839895e" containerName="payload-processing"
Sep 19 05:54:09.630677 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.630661    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.633210 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.633191    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-0c7c7a36-gw-openshift-default-dockercfg-l2tmd\""
Sep 19 05:54:09.643648 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.643625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"]
Sep 19 05:54:09.680609 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45710669-eb94-4019-970c-c6a00d559ce8-istio-podinfo\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680732 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-envoy\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680732 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-data\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680732 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-credential-socket\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680732 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45710669-eb94-4019-970c-c6a00d559ce8-istiod-ca-cert\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680732 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-certs\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680910 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-socket\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680910 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-istio-token\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.680910 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.680831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-484fw\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-kube-api-access-484fw\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782174 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-socket\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-istio-token\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-484fw\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-kube-api-access-484fw\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45710669-eb94-4019-970c-c6a00d559ce8-istio-podinfo\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-envoy\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-data\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782359 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-credential-socket\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782678 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45710669-eb94-4019-970c-c6a00d559ce8-istiod-ca-cert\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782678 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-certs\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-socket\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-data\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.782899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-credential-socket\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.783008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.782926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-certs\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.783890 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.783863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45710669-eb94-4019-970c-c6a00d559ce8-istiod-ca-cert\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.784884 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.784860    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-envoy\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.784983 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.784913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45710669-eb94-4019-970c-c6a00d559ce8-istio-podinfo\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.790194 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.790167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-istio-token\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.790703 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.790683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-484fw\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-kube-api-access-484fw\") pod \"e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") " pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:09.941687 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:09.941614    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:10.218408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:10.218385    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"]
Sep 19 05:54:10.220012 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:54:10.219983    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45710669_eb94_4019_970c_c6a00d559ce8.slice/crio-e1d209c56abef4da3a23901e3ab8d010f184462c6451f9fe7ce1847909c70966 WatchSource:0}: Error finding container e1d209c56abef4da3a23901e3ab8d010f184462c6451f9fe7ce1847909c70966: Status 404 returned error can't find the container with id e1d209c56abef4da3a23901e3ab8d010f184462c6451f9fe7ce1847909c70966
Sep 19 05:54:10.222433 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:10.222403    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:10.222524 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:10.222473    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:10.222524 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:10.222507    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:10.478263 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:10.478188    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0de6f50-cdd9-4fe9-960f-140c8839895e" path="/var/lib/kubelet/pods/a0de6f50-cdd9-4fe9-960f-140c8839895e/volumes"
Sep 19 05:54:11.032931 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:11.032897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" event={"ID":"45710669-eb94-4019-970c-c6a00d559ce8","Type":"ContainerStarted","Data":"ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"}
Sep 19 05:54:11.033391 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:11.032952    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" event={"ID":"45710669-eb94-4019-970c-c6a00d559ce8","Type":"ContainerStarted","Data":"e1d209c56abef4da3a23901e3ab8d010f184462c6451f9fe7ce1847909c70966"}
Sep 19 05:54:11.092852 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:11.092799    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podStartSLOduration=2.092783376 podStartE2EDuration="2.092783376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:11.091823444 +0000 UTC m=+796.702503443" watchObservedRunningTime="2026-09-19 05:54:11.092783376 +0000 UTC m=+796.703463419"
Sep 19 05:54:11.942111 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:11.942078    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:11.947289 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:11.947268    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:12.038008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:12.037983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:12.038937 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:12.038916    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:13.989361 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:13.989330    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"]
Sep 19 05:54:13.993582 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:13.993560    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:13.996470 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:13.996452    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0c7c7a36-serving-cert\""
Sep 19 05:54:14.003635 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.003609    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"]
Sep 19 05:54:14.123751 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.123719    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj5zk\" (UniqueName: \"kubernetes.io/projected/a2c74621-a3cc-4169-8e16-05ea42e7bc53-kube-api-access-qj5zk\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.123942 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.123762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-metrics-certs\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.123942 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.123903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c74621-a3cc-4169-8e16-05ea42e7bc53-openshift-service-ca\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.124105 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.123972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.224433 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.224404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.224583 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.224450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj5zk\" (UniqueName: \"kubernetes.io/projected/a2c74621-a3cc-4169-8e16-05ea42e7bc53-kube-api-access-qj5zk\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.224583 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.224477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-metrics-certs\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.224583 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.224519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c74621-a3cc-4169-8e16-05ea42e7bc53-openshift-service-ca\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.224583 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:54:14.224563    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-serving-cert: secret "maas-api-e2e-migrate-0c7c7a36-serving-cert" not found
Sep 19 05:54:14.224723 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:54:14.224639    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls podName:a2c74621-a3cc-4169-8e16-05ea42e7bc53 nodeName:}" failed. No retries permitted until 2026-09-19 05:54:14.724618961 +0000 UTC m=+800.335298922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls") pod "maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" (UID: "a2c74621-a3cc-4169-8e16-05ea42e7bc53") : secret "maas-api-e2e-migrate-0c7c7a36-serving-cert" not found
Sep 19 05:54:14.225144 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.225126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c74621-a3cc-4169-8e16-05ea42e7bc53-openshift-service-ca\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.227112 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.227092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-metrics-certs\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.234953 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.234929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj5zk\" (UniqueName: \"kubernetes.io/projected/a2c74621-a3cc-4169-8e16-05ea42e7bc53-kube-api-access-qj5zk\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.355608 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.355563    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:54:14.711314 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.711237    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"]
Sep 19 05:54:14.715154 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.715121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:14.717922 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.717897    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0c7c7a36\""
Sep 19 05:54:14.718079 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.717964    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0c7c7a36-dockercfg-ndw2l\""
Sep 19 05:54:14.727602 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.727145    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"]
Sep 19 05:54:14.728842 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.728820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.737992 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.736441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls\") pod \"maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.829694 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.829643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2rmp\" (UniqueName: \"kubernetes.io/projected/4d070553-3d25-4c6e-b677-32998158203a-kube-api-access-g2rmp\") pod \"payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:14.829694 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.829697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d070553-3d25-4c6e-b677-32998158203a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:14.907301 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.907270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:14.930647 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.930615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rmp\" (UniqueName: \"kubernetes.io/projected/4d070553-3d25-4c6e-b677-32998158203a-kube-api-access-g2rmp\") pod \"payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:14.930787 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.930654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d070553-3d25-4c6e-b677-32998158203a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:14.931395 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.931368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d070553-3d25-4c6e-b677-32998158203a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:14.938679 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:14.938654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rmp\" (UniqueName: \"kubernetes.io/projected/4d070553-3d25-4c6e-b677-32998158203a-kube-api-access-g2rmp\") pod \"payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:15.029209 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:15.029174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:15.185875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:15.185847    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"]
Sep 19 05:54:15.187647 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:54:15.187620    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c74621_a3cc_4169_8e16_05ea42e7bc53.slice/crio-1c98547b0ec5de1889f9e25ba0e2cbbcc17d5908a723c76a65752fb82ad94db2 WatchSource:0}: Error finding container 1c98547b0ec5de1889f9e25ba0e2cbbcc17d5908a723c76a65752fb82ad94db2: Status 404 returned error can't find the container with id 1c98547b0ec5de1889f9e25ba0e2cbbcc17d5908a723c76a65752fb82ad94db2
Sep 19 05:54:15.301374 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:15.301350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"]
Sep 19 05:54:15.303357 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:54:15.303325    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d070553_3d25_4c6e_b677_32998158203a.slice/crio-9df22e056dd29b253999c086f7a326ce88806c1ed94fcd1db5a0016992914975 WatchSource:0}: Error finding container 9df22e056dd29b253999c086f7a326ce88806c1ed94fcd1db5a0016992914975: Status 404 returned error can't find the container with id 9df22e056dd29b253999c086f7a326ce88806c1ed94fcd1db5a0016992914975
Sep 19 05:54:16.058501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.058465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" event={"ID":"a2c74621-a3cc-4169-8e16-05ea42e7bc53","Type":"ContainerStarted","Data":"20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed"}
Sep 19 05:54:16.058930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.058523    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:16.058930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.058545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" event={"ID":"a2c74621-a3cc-4169-8e16-05ea42e7bc53","Type":"ContainerStarted","Data":"1c98547b0ec5de1889f9e25ba0e2cbbcc17d5908a723c76a65752fb82ad94db2"}
Sep 19 05:54:16.060126 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.060100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx" event={"ID":"4d070553-3d25-4c6e-b677-32998158203a","Type":"ContainerStarted","Data":"d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860"}
Sep 19 05:54:16.060210 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.060142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx" event={"ID":"4d070553-3d25-4c6e-b677-32998158203a","Type":"ContainerStarted","Data":"9df22e056dd29b253999c086f7a326ce88806c1ed94fcd1db5a0016992914975"}
Sep 19 05:54:16.060210 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.060186    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:16.080045 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.079993    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" podStartSLOduration=2.7332012260000003 podStartE2EDuration="3.079981237s" totalImagesPullingTime="346.780011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:16.077379251 +0000 UTC m=+801.688059227" watchObservedRunningTime="2026-09-19 05:54:16.079981237 +0000 UTC m=+801.690661201"
Sep 19 05:54:16.095199 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:16.095159    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx" podStartSLOduration=2.0951447 podStartE2EDuration="2.0951447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:16.093878925 +0000 UTC m=+801.704558890" watchObservedRunningTime="2026-09-19 05:54:16.0951447 +0000 UTC m=+801.705824664"
Sep 19 05:54:22.071626 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:22.071593    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:27.068477 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:27.068440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:31.245874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:31.245836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:54:33.161059 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:33.161001    2581 generic.go:356] "Generic (PLEG): container finished" podID="45710669-eb94-4019-970c-c6a00d559ce8" containerID="ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95" exitCode=0
Sep 19 05:54:33.161455 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:33.161074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" event={"ID":"45710669-eb94-4019-970c-c6a00d559ce8","Type":"ContainerDied","Data":"ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"}
Sep 19 05:54:33.161509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:33.161456    2581 scope.go:117] "RemoveContainer" containerID="ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"
Sep 19 05:54:33.162599 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:33.162575    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:33.162668 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:33.162638    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:33.162710 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:33.162670    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:54:34.168944 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:34.168912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" event={"ID":"45710669-eb94-4019-970c-c6a00d559ce8","Type":"ContainerStarted","Data":"9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e"}
Sep 19 05:54:34.941924 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:34.941888    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:34.942134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:34.942028    2581 patch_prober.go:28] interesting pod/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 19 05:54:34.942134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:34.942119    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 19 05:54:35.942532 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:35.942488    2581 patch_prober.go:28] interesting pod/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 19 05:54:35.942952 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:35.942560    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 19 05:54:36.764679 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.764644    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"]
Sep 19 05:54:36.768266 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.768248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.771280 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.771260    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"kube-root-ca.crt\""
Sep 19 05:54:36.772512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.772492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"default-dockercfg-2h5kn\""
Sep 19 05:54:36.772605 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.772526    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"simulator-w4-3182b5-kserve-self-signed-certs\""
Sep 19 05:54:36.772605 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.772526    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"openshift-service-ca.crt\""
Sep 19 05:54:36.777644 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.777620    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"]
Sep 19 05:54:36.938584 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.938547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-model-cache\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.938757 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.938597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tls-certs\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.938757 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.938628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-494d7\" (UniqueName: \"kubernetes.io/projected/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-kube-api-access-494d7\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.938757 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.938655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-home\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.938889 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.938761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-dshm\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.938889 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.938786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tmp-dir\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:36.942614 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.942596    2581 patch_prober.go:28] interesting pod/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 19 05:54:36.942898 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:36.942648    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 19 05:54:37.039630 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.039598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-model-cache\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.039828 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.039638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tls-certs\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.039828 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.039659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-494d7\" (UniqueName: \"kubernetes.io/projected/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-kube-api-access-494d7\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.039828 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.039778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-home\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.040020 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.039871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-dshm\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.040020 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.039893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tmp-dir\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.040154 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.040116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-model-cache\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.040196 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.040153    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-home\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.040288 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.040265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tmp-dir\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.042221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.042204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-dshm\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.042484 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.042467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tls-certs\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.049357 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.049331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-494d7\" (UniqueName: \"kubernetes.io/projected/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-kube-api-access-494d7\") pod \"simulator-w4-3182b5-kserve-7877966c8d-nh9n8\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") " pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.078875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.078854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:37.375824 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.375798    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"]
Sep 19 05:54:37.377380 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:54:37.377348    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cd54113_b6c3_4ecf_8217_e5f211a9d5b7.slice/crio-34ebf313fc1dc2a3bb529ac7c7d972c16186029803c504da8bf23f24f34541e9 WatchSource:0}: Error finding container 34ebf313fc1dc2a3bb529ac7c7d972c16186029803c504da8bf23f24f34541e9: Status 404 returned error can't find the container with id 34ebf313fc1dc2a3bb529ac7c7d972c16186029803c504da8bf23f24f34541e9
Sep 19 05:54:37.941953 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.941920    2581 patch_prober.go:28] interesting pod/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 19 05:54:37.942137 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:37.941972    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 19 05:54:38.201799 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:38.201700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8" event={"ID":"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7","Type":"ContainerStarted","Data":"d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e"}
Sep 19 05:54:38.201799 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:38.201760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8" event={"ID":"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7","Type":"ContainerStarted","Data":"34ebf313fc1dc2a3bb529ac7c7d972c16186029803c504da8bf23f24f34541e9"}
Sep 19 05:54:38.218536 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:38.218488    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8" podStartSLOduration=2.218463041 podStartE2EDuration="2.218463041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:38.218304763 +0000 UTC m=+823.828984731" watchObservedRunningTime="2026-09-19 05:54:38.218463041 +0000 UTC m=+823.829143006"
Sep 19 05:54:38.942535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:38.942495    2581 patch_prober.go:28] interesting pod/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 19 05:54:38.942697 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:38.942550    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 19 05:54:39.946241 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:39.946209    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:39.946611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:39.946548    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:39.947475 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:39.947452    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:54:44.160357 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:44.160321    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:54:47.079275 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:47.079239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:47.079275 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:47.079285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:47.092611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:47.092578    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:47.260273 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:47.260247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 05:54:50.032756 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.032716    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"]
Sep 19 05:54:50.037659 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.037638    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.040509 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.040487    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"openshift-service-ca.crt\""
Sep 19 05:54:50.041527 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.041505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"default-dockercfg-mwx7m\""
Sep 19 05:54:50.041652 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.041555    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"kube-root-ca.crt\""
Sep 19 05:54:50.041652 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.041604    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"simulator-w0-cc0e26-kserve-self-signed-certs\""
Sep 19 05:54:50.044580 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.044557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"]
Sep 19 05:54:50.156003 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.155968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-tmp-dir\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.156226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.156027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skwnd\" (UniqueName: \"kubernetes.io/projected/c6546b48-1549-41e5-ad72-19ea7c3781d4-kube-api-access-skwnd\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.156226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.156093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-dshm\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.156226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.156122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6546b48-1549-41e5-ad72-19ea7c3781d4-tls-certs\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.156226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.156197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-model-cache\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.156408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.156291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-home\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257280 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-home\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257472 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-tmp-dir\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257472 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwnd\" (UniqueName: \"kubernetes.io/projected/c6546b48-1549-41e5-ad72-19ea7c3781d4-kube-api-access-skwnd\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257472 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-dshm\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257472 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6546b48-1549-41e5-ad72-19ea7c3781d4-tls-certs\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257472 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-model-cache\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257704    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-home\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257718    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-tmp-dir\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.257872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.257852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-model-cache\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.259832 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.259801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-dshm\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.260000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.259984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6546b48-1549-41e5-ad72-19ea7c3781d4-tls-certs\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.270971 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.270940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skwnd\" (UniqueName: \"kubernetes.io/projected/c6546b48-1549-41e5-ad72-19ea7c3781d4-kube-api-access-skwnd\") pod \"simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") " pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.348133 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.348054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:54:50.621712 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:50.621685    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"]
Sep 19 05:54:50.623484 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:54:50.623455    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6546b48_1549_41e5_ad72_19ea7c3781d4.slice/crio-93b54fd177e5919e46d551e32343c2c12087fb57e5ee93a3a2fda92b93269110 WatchSource:0}: Error finding container 93b54fd177e5919e46d551e32343c2c12087fb57e5ee93a3a2fda92b93269110: Status 404 returned error can't find the container with id 93b54fd177e5919e46d551e32343c2c12087fb57e5ee93a3a2fda92b93269110
Sep 19 05:54:51.278000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:51.277970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm" event={"ID":"c6546b48-1549-41e5-ad72-19ea7c3781d4","Type":"ContainerStarted","Data":"f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387"}
Sep 19 05:54:51.278371 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:51.278009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm" event={"ID":"c6546b48-1549-41e5-ad72-19ea7c3781d4","Type":"ContainerStarted","Data":"93b54fd177e5919e46d551e32343c2c12087fb57e5ee93a3a2fda92b93269110"}
Sep 19 05:54:51.299993 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:51.299933    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm" podStartSLOduration=1.299915008 podStartE2EDuration="1.299915008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:51.297512292 +0000 UTC m=+836.908192281" watchObservedRunningTime="2026-09-19 05:54:51.299915008 +0000 UTC m=+836.910594973"
Sep 19 05:54:54.300720 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.300688    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"]
Sep 19 05:54:54.301236 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.300899    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" podUID="a2c74621-a3cc-4169-8e16-05ea42e7bc53" containerName="maas-api" containerID="cri-o://20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed" gracePeriod=30
Sep 19 05:54:54.361145 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.361115    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"]
Sep 19 05:54:54.361346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.361317    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx" podUID="4d070553-3d25-4c6e-b677-32998158203a" containerName="payload-pre-processing" containerID="cri-o://d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860" gracePeriod=30
Sep 19 05:54:54.661475 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.661444    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"]
Sep 19 05:54:54.661878 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.661842    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" containerID="cri-o://9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e" gracePeriod=30
Sep 19 05:54:54.686109 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.686017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:54.702553 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.702527    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:54.810008 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.809972    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls\") pod \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") "
Sep 19 05:54:54.810213 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-metrics-certs\") pod \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") "
Sep 19 05:54:54.810213 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810126    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj5zk\" (UniqueName: \"kubernetes.io/projected/a2c74621-a3cc-4169-8e16-05ea42e7bc53-kube-api-access-qj5zk\") pod \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") "
Sep 19 05:54:54.810213 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c74621-a3cc-4169-8e16-05ea42e7bc53-openshift-service-ca\") pod \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\" (UID: \"a2c74621-a3cc-4169-8e16-05ea42e7bc53\") "
Sep 19 05:54:54.810372 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810223    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2rmp\" (UniqueName: \"kubernetes.io/projected/4d070553-3d25-4c6e-b677-32998158203a-kube-api-access-g2rmp\") pod \"4d070553-3d25-4c6e-b677-32998158203a\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") "
Sep 19 05:54:54.810372 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810270    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d070553-3d25-4c6e-b677-32998158203a-plugins-config-volume\") pod \"4d070553-3d25-4c6e-b677-32998158203a\" (UID: \"4d070553-3d25-4c6e-b677-32998158203a\") "
Sep 19 05:54:54.810559 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810526    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c74621-a3cc-4169-8e16-05ea42e7bc53-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2c74621-a3cc-4169-8e16-05ea42e7bc53" (UID: "a2c74621-a3cc-4169-8e16-05ea42e7bc53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:54:54.810783 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.810709    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d070553-3d25-4c6e-b677-32998158203a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d070553-3d25-4c6e-b677-32998158203a" (UID: "4d070553-3d25-4c6e-b677-32998158203a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:54:54.812517 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.812488    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2c74621-a3cc-4169-8e16-05ea42e7bc53" (UID: "a2c74621-a3cc-4169-8e16-05ea42e7bc53"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:54:54.812604 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.812559    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c74621-a3cc-4169-8e16-05ea42e7bc53-kube-api-access-qj5zk" (OuterVolumeSpecName: "kube-api-access-qj5zk") pod "a2c74621-a3cc-4169-8e16-05ea42e7bc53" (UID: "a2c74621-a3cc-4169-8e16-05ea42e7bc53"). InnerVolumeSpecName "kube-api-access-qj5zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:54:54.812868 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.812848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d070553-3d25-4c6e-b677-32998158203a-kube-api-access-g2rmp" (OuterVolumeSpecName: "kube-api-access-g2rmp") pod "4d070553-3d25-4c6e-b677-32998158203a" (UID: "4d070553-3d25-4c6e-b677-32998158203a"). InnerVolumeSpecName "kube-api-access-g2rmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:54:54.812945 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.812849    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2c74621-a3cc-4169-8e16-05ea42e7bc53" (UID: "a2c74621-a3cc-4169-8e16-05ea42e7bc53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:54:54.911408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.911328    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c74621-a3cc-4169-8e16-05ea42e7bc53-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:54.911408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.911356    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2rmp\" (UniqueName: \"kubernetes.io/projected/4d070553-3d25-4c6e-b677-32998158203a-kube-api-access-g2rmp\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:54.911408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.911366    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d070553-3d25-4c6e-b677-32998158203a-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:54.911408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.911374    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:54.911408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.911383    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c74621-a3cc-4169-8e16-05ea42e7bc53-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:54.911408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.911391    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj5zk\" (UniqueName: \"kubernetes.io/projected/a2c74621-a3cc-4169-8e16-05ea42e7bc53-kube-api-access-qj5zk\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:54:54.949789 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.949755    2581 patch_prober.go:28] interesting pod/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 05:54:54.949922 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:54.949811    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:54:55.306554 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.306502    2581 generic.go:356] "Generic (PLEG): container finished" podID="4d070553-3d25-4c6e-b677-32998158203a" containerID="d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860" exitCode=0
Sep 19 05:54:55.306554 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.306548    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"
Sep 19 05:54:55.307083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.306635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx" event={"ID":"4d070553-3d25-4c6e-b677-32998158203a","Type":"ContainerDied","Data":"d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860"}
Sep 19 05:54:55.307083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.306698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx" event={"ID":"4d070553-3d25-4c6e-b677-32998158203a","Type":"ContainerDied","Data":"9df22e056dd29b253999c086f7a326ce88806c1ed94fcd1db5a0016992914975"}
Sep 19 05:54:55.307083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.306727    2581 scope.go:117] "RemoveContainer" containerID="d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860"
Sep 19 05:54:55.308298 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.308274    2581 generic.go:356] "Generic (PLEG): container finished" podID="a2c74621-a3cc-4169-8e16-05ea42e7bc53" containerID="20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed" exitCode=0
Sep 19 05:54:55.308573 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.308313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" event={"ID":"a2c74621-a3cc-4169-8e16-05ea42e7bc53","Type":"ContainerDied","Data":"20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed"}
Sep 19 05:54:55.308573 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.308343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp" event={"ID":"a2c74621-a3cc-4169-8e16-05ea42e7bc53","Type":"ContainerDied","Data":"1c98547b0ec5de1889f9e25ba0e2cbbcc17d5908a723c76a65752fb82ad94db2"}
Sep 19 05:54:55.308573 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.308348    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"
Sep 19 05:54:55.324163 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.324147    2581 scope.go:117] "RemoveContainer" containerID="d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860"
Sep 19 05:54:55.324421 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:54:55.324396    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860\": container with ID starting with d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860 not found: ID does not exist" containerID="d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860"
Sep 19 05:54:55.324482 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.324430    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860"} err="failed to get container status \"d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860\": rpc error: code = NotFound desc = could not find container \"d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860\": container with ID starting with d3a30531cc75b2c083f80ba434ec5685770229f21af292c8356e0234bcfd4860 not found: ID does not exist"
Sep 19 05:54:55.324482 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.324447    2581 scope.go:117] "RemoveContainer" containerID="20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed"
Sep 19 05:54:55.332620 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.332604    2581 scope.go:117] "RemoveContainer" containerID="20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed"
Sep 19 05:54:55.332897 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:54:55.332880    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed\": container with ID starting with 20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed not found: ID does not exist" containerID="20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed"
Sep 19 05:54:55.332939 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.332905    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed"} err="failed to get container status \"20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed\": rpc error: code = NotFound desc = could not find container \"20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed\": container with ID starting with 20f66e7ee0ddb8aab02ed026a447f989913a4c8e0af997bb92b2d34195b085ed not found: ID does not exist"
Sep 19 05:54:55.344243 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.344170    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"]
Sep 19 05:54:55.346830 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.346809    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0c7c7a36-5dc9b7d49b-r78qp"]
Sep 19 05:54:55.355729 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.355708    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"]
Sep 19 05:54:55.362022 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:55.362004    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0c7c7a36-5cff67b878-vx2hx"]
Sep 19 05:54:56.478410 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:56.478383    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d070553-3d25-4c6e-b677-32998158203a" path="/var/lib/kubelet/pods/4d070553-3d25-4c6e-b677-32998158203a/volumes"
Sep 19 05:54:56.478793 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:56.478777    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c74621-a3cc-4169-8e16-05ea42e7bc53" path="/var/lib/kubelet/pods/a2c74621-a3cc-4169-8e16-05ea42e7bc53/volumes"
Sep 19 05:54:59.918621 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:54:59.918592    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:55:00.057118 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057080    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45710669-eb94-4019-970c-c6a00d559ce8-istiod-ca-cert\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057126    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-data\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057143    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-credential-socket\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057168    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-certs\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057189    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-envoy\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057207    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-484fw\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-kube-api-access-484fw\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057232    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45710669-eb94-4019-970c-c6a00d559ce8-istio-podinfo\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057262    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-socket\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057286    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-istio-token\") pod \"45710669-eb94-4019-970c-c6a00d559ce8\" (UID: \"45710669-eb94-4019-970c-c6a00d559ce8\") "
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-data" (OuterVolumeSpecName: "istio-data") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057465    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057520    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057590    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45710669-eb94-4019-970c-c6a00d559ce8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057676    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-data\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057736    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-credential-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057752    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.057796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.057648    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:00.059714 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.059689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-kube-api-access-484fw" (OuterVolumeSpecName: "kube-api-access-484fw") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "kube-api-access-484fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:00.059831 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.059687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45710669-eb94-4019-970c-c6a00d559ce8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 05:55:00.060158 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.060126    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:00.060158 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.060133    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-istio-token" (OuterVolumeSpecName: "istio-token") pod "45710669-eb94-4019-970c-c6a00d559ce8" (UID: "45710669-eb94-4019-970c-c6a00d559ce8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:00.158998 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.158962    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45710669-eb94-4019-970c-c6a00d559ce8-istiod-ca-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.158998 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.158990    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-istio-envoy\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.158998 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.159002    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-484fw\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-kube-api-access-484fw\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.159284 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.159011    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45710669-eb94-4019-970c-c6a00d559ce8-istio-podinfo\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.159284 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.159020    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45710669-eb94-4019-970c-c6a00d559ce8-workload-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.159284 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.159087    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45710669-eb94-4019-970c-c6a00d559ce8-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:00.337656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.337568    2581 generic.go:356] "Generic (PLEG): container finished" podID="45710669-eb94-4019-970c-c6a00d559ce8" containerID="9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e" exitCode=0
Sep 19 05:55:00.337656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.337615    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"
Sep 19 05:55:00.337656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.337645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" event={"ID":"45710669-eb94-4019-970c-c6a00d559ce8","Type":"ContainerDied","Data":"9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e"}
Sep 19 05:55:00.338106 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.337697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt" event={"ID":"45710669-eb94-4019-970c-c6a00d559ce8","Type":"ContainerDied","Data":"e1d209c56abef4da3a23901e3ab8d010f184462c6451f9fe7ce1847909c70966"}
Sep 19 05:55:00.338106 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.337712    2581 scope.go:117] "RemoveContainer" containerID="9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e"
Sep 19 05:55:00.347504 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.347485    2581 scope.go:117] "RemoveContainer" containerID="ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"
Sep 19 05:55:00.348204 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.348186    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:55:00.348412 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.348385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:55:00.360331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.360308    2581 scope.go:117] "RemoveContainer" containerID="9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e"
Sep 19 05:55:00.360628 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:55:00.360603    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e\": container with ID starting with 9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e not found: ID does not exist" containerID="9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e"
Sep 19 05:55:00.360709 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.360640    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e"} err="failed to get container status \"9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e\": rpc error: code = NotFound desc = could not find container \"9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e\": container with ID starting with 9239a6b9eca16e3b20f89b93dbaf4b84dfb49a29de57dcd613ad9a434466cd8e not found: ID does not exist"
Sep 19 05:55:00.360709 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.360664    2581 scope.go:117] "RemoveContainer" containerID="ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"
Sep 19 05:55:00.360917 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:55:00.360900    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95\": container with ID starting with ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95 not found: ID does not exist" containerID="ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"
Sep 19 05:55:00.360962 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.360925    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95"} err="failed to get container status \"ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95\": rpc error: code = NotFound desc = could not find container \"ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95\": container with ID starting with ef3edf6ea0fd511b2123653943ca996dad5a8b02d782373cccfd1fff7e919f95 not found: ID does not exist"
Sep 19 05:55:00.361167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.361150    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"]
Sep 19 05:55:00.362939 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.362918    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:55:00.365062 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.365024    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-0c7c7a36-gw-openshift-default-7895b4fb64-7zwlt"]
Sep 19 05:55:00.455299 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.455273    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:55:00.479650 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.479625    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45710669-eb94-4019-970c-c6a00d559ce8" path="/var/lib/kubelet/pods/45710669-eb94-4019-970c-c6a00d559ce8/volumes"
Sep 19 05:55:00.486305 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486280    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"]
Sep 19 05:55:00.486666 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486653    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy"
Sep 19 05:55:00.486718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486668    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy"
Sep 19 05:55:00.486718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486683    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d070553-3d25-4c6e-b677-32998158203a" containerName="payload-pre-processing"
Sep 19 05:55:00.486718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486691    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d070553-3d25-4c6e-b677-32998158203a" containerName="payload-pre-processing"
Sep 19 05:55:00.486718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486708    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy"
Sep 19 05:55:00.486718 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486713    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy"
Sep 19 05:55:00.486917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486730    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c74621-a3cc-4169-8e16-05ea42e7bc53" containerName="maas-api"
Sep 19 05:55:00.486917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486735    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c74621-a3cc-4169-8e16-05ea42e7bc53" containerName="maas-api"
Sep 19 05:55:00.486917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486795    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy"
Sep 19 05:55:00.486917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486802    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d070553-3d25-4c6e-b677-32998158203a" containerName="payload-pre-processing"
Sep 19 05:55:00.486917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.486811    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c74621-a3cc-4169-8e16-05ea42e7bc53" containerName="maas-api"
Sep 19 05:55:00.491375 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.491304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.495060 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.494701    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-32a87fd8-serving-cert\""
Sep 19 05:55:00.502002 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.501976    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"]
Sep 19 05:55:00.562646 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.562617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-metrics-certs\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.562784 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.562756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thnsm\" (UniqueName: \"kubernetes.io/projected/66e7f59b-dfd3-4619-b86c-52bdf263812d-kube-api-access-thnsm\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.563135 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.563101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-maas-api-tls\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.563232 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.563212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7f59b-dfd3-4619-b86c-52bdf263812d-openshift-service-ca\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.664371 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.664301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thnsm\" (UniqueName: \"kubernetes.io/projected/66e7f59b-dfd3-4619-b86c-52bdf263812d-kube-api-access-thnsm\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.664371 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.664346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-maas-api-tls\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.664590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.664379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7f59b-dfd3-4619-b86c-52bdf263812d-openshift-service-ca\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.664590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.664408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-metrics-certs\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.665210 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.665121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7f59b-dfd3-4619-b86c-52bdf263812d-openshift-service-ca\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.666917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.666896    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-metrics-certs\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.667000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.666946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-maas-api-tls\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.675988 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.675959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thnsm\" (UniqueName: \"kubernetes.io/projected/66e7f59b-dfd3-4619-b86c-52bdf263812d-kube-api-access-thnsm\") pod \"maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:00.806964 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:00.806926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:01.082104 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.082081    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"]
Sep 19 05:55:01.083646 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:55:01.083617    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e7f59b_dfd3_4619_b86c_52bdf263812d.slice/crio-3ffa8bb791637639a0a4da80b1408943e16cbcd4f1b045487b94942dd54cb13d WatchSource:0}: Error finding container 3ffa8bb791637639a0a4da80b1408943e16cbcd4f1b045487b94942dd54cb13d: Status 404 returned error can't find the container with id 3ffa8bb791637639a0a4da80b1408943e16cbcd4f1b045487b94942dd54cb13d
Sep 19 05:55:01.344916 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.344830    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc" event={"ID":"66e7f59b-dfd3-4619-b86c-52bdf263812d","Type":"ContainerStarted","Data":"3ffa8bb791637639a0a4da80b1408943e16cbcd4f1b045487b94942dd54cb13d"}
Sep 19 05:55:01.356954 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.356928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 05:55:01.458168 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.458132    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"]
Sep 19 05:55:01.458568 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.458555    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45710669-eb94-4019-970c-c6a00d559ce8" containerName="istio-proxy"
Sep 19 05:55:01.461784 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.461770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.464874 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.464851    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-32a87fd8-dockercfg-7ppds\""
Sep 19 05:55:01.464996 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.464923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-32a87fd8\""
Sep 19 05:55:01.472467 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.472323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"]
Sep 19 05:55:01.573352 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.573324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05f433c-4381-42d5-918c-e0985d376900-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.573475 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.573433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mmxp\" (UniqueName: \"kubernetes.io/projected/c05f433c-4381-42d5-918c-e0985d376900-kube-api-access-7mmxp\") pod \"payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.674341 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.674261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05f433c-4381-42d5-918c-e0985d376900-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.674341 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.674335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mmxp\" (UniqueName: \"kubernetes.io/projected/c05f433c-4381-42d5-918c-e0985d376900-kube-api-access-7mmxp\") pod \"payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.674948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.674912    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05f433c-4381-42d5-918c-e0985d376900-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.683088 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.683061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mmxp\" (UniqueName: \"kubernetes.io/projected/c05f433c-4381-42d5-918c-e0985d376900-kube-api-access-7mmxp\") pod \"payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:01.774405 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:01.774372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:02.074786 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.074759    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"]
Sep 19 05:55:02.075661 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:55:02.075635    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05f433c_4381_42d5_918c_e0985d376900.slice/crio-f8acfdcd3cc8f3c89856912915ed66dbf0ace628f8977485349534bcdb581848 WatchSource:0}: Error finding container f8acfdcd3cc8f3c89856912915ed66dbf0ace628f8977485349534bcdb581848: Status 404 returned error can't find the container with id f8acfdcd3cc8f3c89856912915ed66dbf0ace628f8977485349534bcdb581848
Sep 19 05:55:02.352178 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.352091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr" event={"ID":"c05f433c-4381-42d5-918c-e0985d376900","Type":"ContainerStarted","Data":"2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a"}
Sep 19 05:55:02.352178 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.352142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr" event={"ID":"c05f433c-4381-42d5-918c-e0985d376900","Type":"ContainerStarted","Data":"f8acfdcd3cc8f3c89856912915ed66dbf0ace628f8977485349534bcdb581848"}
Sep 19 05:55:02.352656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.352207    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:02.353770 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.353743    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc" event={"ID":"66e7f59b-dfd3-4619-b86c-52bdf263812d","Type":"ContainerStarted","Data":"a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833"}
Sep 19 05:55:02.373197 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.373156    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr" podStartSLOduration=1.373142628 podStartE2EDuration="1.373142628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:02.372833837 +0000 UTC m=+847.983513828" watchObservedRunningTime="2026-09-19 05:55:02.373142628 +0000 UTC m=+847.983822592"
Sep 19 05:55:02.395621 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:02.395578    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc" podStartSLOduration=2.007411 podStartE2EDuration="2.395565466s" totalImagesPullingTime="388.154466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:02.394849464 +0000 UTC m=+848.005529454" watchObservedRunningTime="2026-09-19 05:55:02.395565466 +0000 UTC m=+848.006245430"
Sep 19 05:55:03.359365 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:03.359334    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:09.349806 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.349773    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"]
Sep 19 05:55:09.353868 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.353844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.357827 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.357800    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-8128f3\"/\"openshift-service-ca.crt\""
Sep 19 05:55:09.357961 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.357918    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-8128f3\"/\"kube-root-ca.crt\""
Sep 19 05:55:09.357961 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.357934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-8128f3\"/\"default-dockercfg-wr56f\""
Sep 19 05:55:09.358238 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.358221    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-8128f3\"/\"autores-model-8128f3-kserve-self-signed-certs\""
Sep 19 05:55:09.363298 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.363275    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"]
Sep 19 05:55:09.370553 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.370534    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:09.451216 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.451186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tmp-dir\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.451408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.451300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-dshm\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.451408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.451347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-model-cache\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.451535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.451406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzgff\" (UniqueName: \"kubernetes.io/projected/9c25a189-d50f-4c7c-93ce-ebe3febce5be-kube-api-access-rzgff\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.451535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.451454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-home\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.451535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.451482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tls-certs\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552393 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-home\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552568 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tls-certs\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552568 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tmp-dir\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552568 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-dshm\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552724 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-model-cache\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552724 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzgff\" (UniqueName: \"kubernetes.io/projected/9c25a189-d50f-4c7c-93ce-ebe3febce5be-kube-api-access-rzgff\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552859 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-home\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.552917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.552845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tmp-dir\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.553523 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.553494    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-model-cache\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.554914 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.554889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-dshm\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.555134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.555024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tls-certs\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.561438 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.561417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzgff\" (UniqueName: \"kubernetes.io/projected/9c25a189-d50f-4c7c-93ce-ebe3febce5be-kube-api-access-rzgff\") pod \"autores-model-8128f3-kserve-58cc594f7f-fgfzq\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") " pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.664232 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.664149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:09.941272 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:09.941226    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"]
Sep 19 05:55:09.944293 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:55:09.944259    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c25a189_d50f_4c7c_93ce_ebe3febce5be.slice/crio-17deebdd08d94f4d29d81c92ae97adf8169aabd5cb1a4ae7ea9176be773971f8 WatchSource:0}: Error finding container 17deebdd08d94f4d29d81c92ae97adf8169aabd5cb1a4ae7ea9176be773971f8: Status 404 returned error can't find the container with id 17deebdd08d94f4d29d81c92ae97adf8169aabd5cb1a4ae7ea9176be773971f8
Sep 19 05:55:10.398412 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:10.398369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq" event={"ID":"9c25a189-d50f-4c7c-93ce-ebe3febce5be","Type":"ContainerStarted","Data":"d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633"}
Sep 19 05:55:10.398807 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:10.398429    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq" event={"ID":"9c25a189-d50f-4c7c-93ce-ebe3febce5be","Type":"ContainerStarted","Data":"17deebdd08d94f4d29d81c92ae97adf8169aabd5cb1a4ae7ea9176be773971f8"}
Sep 19 05:55:10.416505 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:10.416331    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq" podStartSLOduration=1.416316237 podStartE2EDuration="1.416316237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:10.415749714 +0000 UTC m=+856.026429714" watchObservedRunningTime="2026-09-19 05:55:10.416316237 +0000 UTC m=+856.026996202"
Sep 19 05:55:13.361165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:13.361136    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:19.664935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:19.664891    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:19.665376 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:19.664958    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:19.677405 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:19.677372    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:19.958864 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:19.958782    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:55:20.458908 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:20.458879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:55:25.350250 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.350213    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"]
Sep 19 05:55:25.353976 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.353950    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.358017 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.357997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4b78e2\"/\"default-dockercfg-jbssm\""
Sep 19 05:55:25.358165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.358002    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4b78e2\"/\"openshift-service-ca.crt\""
Sep 19 05:55:25.358165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.358086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4b78e2\"/\"test-model-4b78e2-kserve-self-signed-certs\""
Sep 19 05:55:25.358165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.358148    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4b78e2\"/\"kube-root-ca.crt\""
Sep 19 05:55:25.362828 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.362810    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"]
Sep 19 05:55:25.397371 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.397338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebe971b-27a5-4447-b147-3eaefbff9f29-tls-certs\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.397371 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.397372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-dshm\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.397579 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.397406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-model-cache\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.397579 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.397430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-tmp-dir\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.397579 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.397509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjpdd\" (UniqueName: \"kubernetes.io/projected/aebe971b-27a5-4447-b147-3eaefbff9f29-kube-api-access-gjpdd\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.397681 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.397619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-home\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499133 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebe971b-27a5-4447-b147-3eaefbff9f29-tls-certs\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499133 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-dshm\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-model-cache\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-tmp-dir\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjpdd\" (UniqueName: \"kubernetes.io/projected/aebe971b-27a5-4447-b147-3eaefbff9f29-kube-api-access-gjpdd\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499549 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-home\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499758 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499729    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-tmp-dir\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499883 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-home\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.499957 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.499916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-model-cache\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.501800 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.501772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-dshm\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.501930 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.501914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebe971b-27a5-4447-b147-3eaefbff9f29-tls-certs\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.508505 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.508484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjpdd\" (UniqueName: \"kubernetes.io/projected/aebe971b-27a5-4447-b147-3eaefbff9f29-kube-api-access-gjpdd\") pod \"test-model-4b78e2-kserve-74fb9cdb49-27mm5\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") " pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.664812 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.664715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:25.941462 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:25.941434    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"]
Sep 19 05:55:25.943519 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:55:25.943493    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaebe971b_27a5_4447_b147_3eaefbff9f29.slice/crio-c3964471ca357966de00900f0d14faa7e330e53ee402e30b646897ab9f933430 WatchSource:0}: Error finding container c3964471ca357966de00900f0d14faa7e330e53ee402e30b646897ab9f933430: Status 404 returned error can't find the container with id c3964471ca357966de00900f0d14faa7e330e53ee402e30b646897ab9f933430
Sep 19 05:55:26.479494 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:26.479455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5" event={"ID":"aebe971b-27a5-4447-b147-3eaefbff9f29","Type":"ContainerStarted","Data":"c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937"}
Sep 19 05:55:26.479494 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:26.479499    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5" event={"ID":"aebe971b-27a5-4447-b147-3eaefbff9f29","Type":"ContainerStarted","Data":"c3964471ca357966de00900f0d14faa7e330e53ee402e30b646897ab9f933430"}
Sep 19 05:55:26.501406 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:26.501336    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5" podStartSLOduration=1.501314916 podStartE2EDuration="1.501314916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:26.496807987 +0000 UTC m=+872.107487953" watchObservedRunningTime="2026-09-19 05:55:26.501314916 +0000 UTC m=+872.111994882"
Sep 19 05:55:35.665290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:35.665243    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:35.665709 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:35.665304    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:35.678119 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:35.678089    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:36.547815 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:36.547736    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 05:55:40.764384 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:40.764348    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"]
Sep 19 05:55:40.764855 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:40.764584    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc" podUID="66e7f59b-dfd3-4619-b86c-52bdf263812d" containerName="maas-api" containerID="cri-o://a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833" gracePeriod=30
Sep 19 05:55:40.826981 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:40.826946    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"]
Sep 19 05:55:40.827259 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:40.827198    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr" podUID="c05f433c-4381-42d5-918c-e0985d376900" containerName="payload-pre-processing" containerID="cri-o://2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a" gracePeriod=30
Sep 19 05:55:41.077705 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.077681    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:41.114535 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.114511    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:41.151290 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.151261    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-metrics-certs\") pod \"66e7f59b-dfd3-4619-b86c-52bdf263812d\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") "
Sep 19 05:55:41.151423 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.151304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7f59b-dfd3-4619-b86c-52bdf263812d-openshift-service-ca\") pod \"66e7f59b-dfd3-4619-b86c-52bdf263812d\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") "
Sep 19 05:55:41.151423 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.151361    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thnsm\" (UniqueName: \"kubernetes.io/projected/66e7f59b-dfd3-4619-b86c-52bdf263812d-kube-api-access-thnsm\") pod \"66e7f59b-dfd3-4619-b86c-52bdf263812d\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") "
Sep 19 05:55:41.151501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.151456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-maas-api-tls\") pod \"66e7f59b-dfd3-4619-b86c-52bdf263812d\" (UID: \"66e7f59b-dfd3-4619-b86c-52bdf263812d\") "
Sep 19 05:55:41.151742 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.151711    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66e7f59b-dfd3-4619-b86c-52bdf263812d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66e7f59b-dfd3-4619-b86c-52bdf263812d" (UID: "66e7f59b-dfd3-4619-b86c-52bdf263812d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:55:41.153596 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.153563    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66e7f59b-dfd3-4619-b86c-52bdf263812d-kube-api-access-thnsm" (OuterVolumeSpecName: "kube-api-access-thnsm") pod "66e7f59b-dfd3-4619-b86c-52bdf263812d" (UID: "66e7f59b-dfd3-4619-b86c-52bdf263812d"). InnerVolumeSpecName "kube-api-access-thnsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:41.153596 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.153573    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "66e7f59b-dfd3-4619-b86c-52bdf263812d" (UID: "66e7f59b-dfd3-4619-b86c-52bdf263812d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:55:41.153722 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.153639    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66e7f59b-dfd3-4619-b86c-52bdf263812d" (UID: "66e7f59b-dfd3-4619-b86c-52bdf263812d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:55:41.252715 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.252683    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mmxp\" (UniqueName: \"kubernetes.io/projected/c05f433c-4381-42d5-918c-e0985d376900-kube-api-access-7mmxp\") pod \"c05f433c-4381-42d5-918c-e0985d376900\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") "
Sep 19 05:55:41.252841 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.252725    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05f433c-4381-42d5-918c-e0985d376900-plugins-config-volume\") pod \"c05f433c-4381-42d5-918c-e0985d376900\" (UID: \"c05f433c-4381-42d5-918c-e0985d376900\") "
Sep 19 05:55:41.252914 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.252900    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e7f59b-dfd3-4619-b86c-52bdf263812d-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:41.252953 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.252916    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thnsm\" (UniqueName: \"kubernetes.io/projected/66e7f59b-dfd3-4619-b86c-52bdf263812d-kube-api-access-thnsm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:41.252953 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.252926    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:41.252953 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.252934    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e7f59b-dfd3-4619-b86c-52bdf263812d-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:41.253193 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.253169    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c05f433c-4381-42d5-918c-e0985d376900-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c05f433c-4381-42d5-918c-e0985d376900" (UID: "c05f433c-4381-42d5-918c-e0985d376900"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:55:41.254885 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.254869    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05f433c-4381-42d5-918c-e0985d376900-kube-api-access-7mmxp" (OuterVolumeSpecName: "kube-api-access-7mmxp") pod "c05f433c-4381-42d5-918c-e0985d376900" (UID: "c05f433c-4381-42d5-918c-e0985d376900"). InnerVolumeSpecName "kube-api-access-7mmxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:41.354172 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.354099    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mmxp\" (UniqueName: \"kubernetes.io/projected/c05f433c-4381-42d5-918c-e0985d376900-kube-api-access-7mmxp\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:41.354172 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.354124    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05f433c-4381-42d5-918c-e0985d376900-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:41.560871 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.560839    2581 generic.go:356] "Generic (PLEG): container finished" podID="c05f433c-4381-42d5-918c-e0985d376900" containerID="2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a" exitCode=0
Sep 19 05:55:41.561063 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.560881    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"
Sep 19 05:55:41.561063 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.560913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr" event={"ID":"c05f433c-4381-42d5-918c-e0985d376900","Type":"ContainerDied","Data":"2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a"}
Sep 19 05:55:41.561063 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.560971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr" event={"ID":"c05f433c-4381-42d5-918c-e0985d376900","Type":"ContainerDied","Data":"f8acfdcd3cc8f3c89856912915ed66dbf0ace628f8977485349534bcdb581848"}
Sep 19 05:55:41.561063 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.560993    2581 scope.go:117] "RemoveContainer" containerID="2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a"
Sep 19 05:55:41.562319 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.562300    2581 generic.go:356] "Generic (PLEG): container finished" podID="66e7f59b-dfd3-4619-b86c-52bdf263812d" containerID="a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833" exitCode=0
Sep 19 05:55:41.562425 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.562340    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"
Sep 19 05:55:41.562425 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.562382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc" event={"ID":"66e7f59b-dfd3-4619-b86c-52bdf263812d","Type":"ContainerDied","Data":"a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833"}
Sep 19 05:55:41.562559 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.562434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc" event={"ID":"66e7f59b-dfd3-4619-b86c-52bdf263812d","Type":"ContainerDied","Data":"3ffa8bb791637639a0a4da80b1408943e16cbcd4f1b045487b94942dd54cb13d"}
Sep 19 05:55:41.570453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.570435    2581 scope.go:117] "RemoveContainer" containerID="2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a"
Sep 19 05:55:41.570758 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:55:41.570738    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a\": container with ID starting with 2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a not found: ID does not exist" containerID="2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a"
Sep 19 05:55:41.570815 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.570765    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a"} err="failed to get container status \"2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a\": rpc error: code = NotFound desc = could not find container \"2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a\": container with ID starting with 2bcf22f2c77a024490c0cc477c0b5ffd40f0f9fbb86be1199fce8237b6bc107a not found: ID does not exist"
Sep 19 05:55:41.570815 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.570781    2581 scope.go:117] "RemoveContainer" containerID="a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833"
Sep 19 05:55:41.578960 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.578943    2581 scope.go:117] "RemoveContainer" containerID="a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833"
Sep 19 05:55:41.579254 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:55:41.579235    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833\": container with ID starting with a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833 not found: ID does not exist" containerID="a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833"
Sep 19 05:55:41.579331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.579261    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833"} err="failed to get container status \"a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833\": rpc error: code = NotFound desc = could not find container \"a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833\": container with ID starting with a0d7d711f0aab6e75c8ee4af1eca61dab4a513bf14e571e2c1e7f61da65ac833 not found: ID does not exist"
Sep 19 05:55:41.585758 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.585736    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"]
Sep 19 05:55:41.589803 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.589783    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-32a87fd8-c4dff8d98-b45kr"]
Sep 19 05:55:41.603404 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.603380    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"]
Sep 19 05:55:41.611147 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:41.611101    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32a87fd8-7b87b6ddfd-6n8jc"]
Sep 19 05:55:42.478339 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:42.478312    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66e7f59b-dfd3-4619-b86c-52bdf263812d" path="/var/lib/kubelet/pods/66e7f59b-dfd3-4619-b86c-52bdf263812d/volumes"
Sep 19 05:55:42.478720 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:42.478704    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c05f433c-4381-42d5-918c-e0985d376900" path="/var/lib/kubelet/pods/c05f433c-4381-42d5-918c-e0985d376900/volumes"
Sep 19 05:55:46.372670 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:46.372636    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:55:47.236007 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:47.235976    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"]
Sep 19 05:55:49.074899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.074864    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"]
Sep 19 05:55:49.075322 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.075265    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66e7f59b-dfd3-4619-b86c-52bdf263812d" containerName="maas-api"
Sep 19 05:55:49.075322 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.075280    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66e7f59b-dfd3-4619-b86c-52bdf263812d" containerName="maas-api"
Sep 19 05:55:49.075322 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.075307    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05f433c-4381-42d5-918c-e0985d376900" containerName="payload-pre-processing"
Sep 19 05:55:49.075322 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.075312    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05f433c-4381-42d5-918c-e0985d376900" containerName="payload-pre-processing"
Sep 19 05:55:49.075460 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.075375    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c05f433c-4381-42d5-918c-e0985d376900" containerName="payload-pre-processing"
Sep 19 05:55:49.075460 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.075387    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="66e7f59b-dfd3-4619-b86c-52bdf263812d" containerName="maas-api"
Sep 19 05:55:49.079822 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.079804    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.082512 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.082490    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-39e34e62\""
Sep 19 05:55:49.082813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.082794    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-39e34e62-dockercfg-7k5sv\""
Sep 19 05:55:49.091822 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.091800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"]
Sep 19 05:55:49.107655 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.107632    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"]
Sep 19 05:55:49.110932 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.110918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.124917 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.124892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"]
Sep 19 05:55:49.236156 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.236124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095676c6-f7ab-46a8-9d17-81cd86063d0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.236318 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.236175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jshzc\" (UniqueName: \"kubernetes.io/projected/033bec5d-16e9-4f56-b2b3-e2f63072b00a-kube-api-access-jshzc\") pod \"payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") " pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.236318 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.236259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/033bec5d-16e9-4f56-b2b3-e2f63072b00a-plugins-config-volume\") pod \"payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") " pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.236391 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.236325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn447\" (UniqueName: \"kubernetes.io/projected/095676c6-f7ab-46a8-9d17-81cd86063d0b-kube-api-access-fn447\") pod \"payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.337594 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.337516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jshzc\" (UniqueName: \"kubernetes.io/projected/033bec5d-16e9-4f56-b2b3-e2f63072b00a-kube-api-access-jshzc\") pod \"payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") " pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.337594 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.337570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/033bec5d-16e9-4f56-b2b3-e2f63072b00a-plugins-config-volume\") pod \"payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") " pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.337792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.337612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn447\" (UniqueName: \"kubernetes.io/projected/095676c6-f7ab-46a8-9d17-81cd86063d0b-kube-api-access-fn447\") pod \"payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.337792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.337677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095676c6-f7ab-46a8-9d17-81cd86063d0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.338586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.338562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/033bec5d-16e9-4f56-b2b3-e2f63072b00a-plugins-config-volume\") pod \"payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") " pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.338652 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.338562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095676c6-f7ab-46a8-9d17-81cd86063d0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.345929 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.345902    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jshzc\" (UniqueName: \"kubernetes.io/projected/033bec5d-16e9-4f56-b2b3-e2f63072b00a-kube-api-access-jshzc\") pod \"payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") " pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.346181 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.346164    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn447\" (UniqueName: \"kubernetes.io/projected/095676c6-f7ab-46a8-9d17-81cd86063d0b-kube-api-access-fn447\") pod \"payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.391614 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.391583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:49.420916 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.420884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:49.669005 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.668979    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"]
Sep 19 05:55:49.670318 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:55:49.670290    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095676c6_f7ab_46a8_9d17_81cd86063d0b.slice/crio-72f3419b70caa0712cb3376a55b73ece7f34d940f39035b279be20abee4ee80f WatchSource:0}: Error finding container 72f3419b70caa0712cb3376a55b73ece7f34d940f39035b279be20abee4ee80f: Status 404 returned error can't find the container with id 72f3419b70caa0712cb3376a55b73ece7f34d940f39035b279be20abee4ee80f
Sep 19 05:55:49.700365 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:49.700339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"]
Sep 19 05:55:49.703107 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:55:49.703084    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033bec5d_16e9_4f56_b2b3_e2f63072b00a.slice/crio-93d1acd63d0be1ea2baa8fc9d5b9df223bdfc3992cd1340cfc2c04597ccd4f30 WatchSource:0}: Error finding container 93d1acd63d0be1ea2baa8fc9d5b9df223bdfc3992cd1340cfc2c04597ccd4f30: Status 404 returned error can't find the container with id 93d1acd63d0be1ea2baa8fc9d5b9df223bdfc3992cd1340cfc2c04597ccd4f30
Sep 19 05:55:50.615611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.615515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc" event={"ID":"033bec5d-16e9-4f56-b2b3-e2f63072b00a","Type":"ContainerStarted","Data":"d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56"}
Sep 19 05:55:50.615611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.615587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc" event={"ID":"033bec5d-16e9-4f56-b2b3-e2f63072b00a","Type":"ContainerStarted","Data":"93d1acd63d0be1ea2baa8fc9d5b9df223bdfc3992cd1340cfc2c04597ccd4f30"}
Sep 19 05:55:50.615611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.615618    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:55:50.617186 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.617163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck" event={"ID":"095676c6-f7ab-46a8-9d17-81cd86063d0b","Type":"ContainerStarted","Data":"a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7"}
Sep 19 05:55:50.617276 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.617210    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck" event={"ID":"095676c6-f7ab-46a8-9d17-81cd86063d0b","Type":"ContainerStarted","Data":"72f3419b70caa0712cb3376a55b73ece7f34d940f39035b279be20abee4ee80f"}
Sep 19 05:55:50.617276 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.617239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:55:50.641116 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.641070    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc" podStartSLOduration=1.64105151 podStartE2EDuration="1.64105151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:50.638863276 +0000 UTC m=+896.249543267" watchObservedRunningTime="2026-09-19 05:55:50.64105151 +0000 UTC m=+896.251731471"
Sep 19 05:55:50.657340 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:50.657295    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck" podStartSLOduration=1.657282417 podStartE2EDuration="1.657282417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:50.654852948 +0000 UTC m=+896.265532969" watchObservedRunningTime="2026-09-19 05:55:50.657282417 +0000 UTC m=+896.267962381"
Sep 19 05:55:52.468858 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.468826    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"]
Sep 19 05:55:52.469263 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.469026    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" podUID="6b4ebd4e-4648-4636-bc98-6aa7b44fab58" containerName="maas-api" containerID="cri-o://d7bfd2883e6ce7141bb11371a4e23c9f57ecfad03d6efe0df7a417c4cb1df201" gracePeriod=30
Sep 19 05:55:52.549288 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.549247    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"]
Sep 19 05:55:52.549510 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.549481    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf" podUID="66c82d38-f61e-4d88-a375-ac037694c6cb" containerName="payload-pre-processing" containerID="cri-o://6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247" gracePeriod=30
Sep 19 05:55:52.630578 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.630548    2581 generic.go:356] "Generic (PLEG): container finished" podID="6b4ebd4e-4648-4636-bc98-6aa7b44fab58" containerID="d7bfd2883e6ce7141bb11371a4e23c9f57ecfad03d6efe0df7a417c4cb1df201" exitCode=0
Sep 19 05:55:52.630727 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.630592    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" event={"ID":"6b4ebd4e-4648-4636-bc98-6aa7b44fab58","Type":"ContainerDied","Data":"d7bfd2883e6ce7141bb11371a4e23c9f57ecfad03d6efe0df7a417c4cb1df201"}
Sep 19 05:55:52.814938 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.814913    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:55:52.844835 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.844811    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:55:52.971885 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.971845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-metrics-certs\") pod \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") "
Sep 19 05:55:52.972070 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.971892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnrj6\" (UniqueName: \"kubernetes.io/projected/66c82d38-f61e-4d88-a375-ac037694c6cb-kube-api-access-pnrj6\") pod \"66c82d38-f61e-4d88-a375-ac037694c6cb\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") "
Sep 19 05:55:52.972070 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.971923    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls\") pod \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") "
Sep 19 05:55:52.972070 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.971959    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66c82d38-f61e-4d88-a375-ac037694c6cb-plugins-config-volume\") pod \"66c82d38-f61e-4d88-a375-ac037694c6cb\" (UID: \"66c82d38-f61e-4d88-a375-ac037694c6cb\") "
Sep 19 05:55:52.972070 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.971983    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-openshift-service-ca\") pod \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") "
Sep 19 05:55:52.972070 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.972002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksvnt\" (UniqueName: \"kubernetes.io/projected/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-kube-api-access-ksvnt\") pod \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\" (UID: \"6b4ebd4e-4648-4636-bc98-6aa7b44fab58\") "
Sep 19 05:55:52.972396 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.972365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c82d38-f61e-4d88-a375-ac037694c6cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66c82d38-f61e-4d88-a375-ac037694c6cb" (UID: "66c82d38-f61e-4d88-a375-ac037694c6cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:55:52.972396 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.972388    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b4ebd4e-4648-4636-bc98-6aa7b44fab58" (UID: "6b4ebd4e-4648-4636-bc98-6aa7b44fab58"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:55:52.974402 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.974379    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b4ebd4e-4648-4636-bc98-6aa7b44fab58" (UID: "6b4ebd4e-4648-4636-bc98-6aa7b44fab58"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:55:52.974402 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.974394    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b4ebd4e-4648-4636-bc98-6aa7b44fab58" (UID: "6b4ebd4e-4648-4636-bc98-6aa7b44fab58"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:55:52.974522 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.974430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c82d38-f61e-4d88-a375-ac037694c6cb-kube-api-access-pnrj6" (OuterVolumeSpecName: "kube-api-access-pnrj6") pod "66c82d38-f61e-4d88-a375-ac037694c6cb" (UID: "66c82d38-f61e-4d88-a375-ac037694c6cb"). InnerVolumeSpecName "kube-api-access-pnrj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:52.974522 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:52.974455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-kube-api-access-ksvnt" (OuterVolumeSpecName: "kube-api-access-ksvnt") pod "6b4ebd4e-4648-4636-bc98-6aa7b44fab58" (UID: "6b4ebd4e-4648-4636-bc98-6aa7b44fab58"). InnerVolumeSpecName "kube-api-access-ksvnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:53.073154 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.073130    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:53.073262 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.073159    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnrj6\" (UniqueName: \"kubernetes.io/projected/66c82d38-f61e-4d88-a375-ac037694c6cb-kube-api-access-pnrj6\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:53.073262 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.073174    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:53.073262 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.073188    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66c82d38-f61e-4d88-a375-ac037694c6cb-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:53.073262 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.073202    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:53.073262 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.073217    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksvnt\" (UniqueName: \"kubernetes.io/projected/6b4ebd4e-4648-4636-bc98-6aa7b44fab58-kube-api-access-ksvnt\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:55:53.636646 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.636617    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"
Sep 19 05:55:53.637089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.636662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s" event={"ID":"6b4ebd4e-4648-4636-bc98-6aa7b44fab58","Type":"ContainerDied","Data":"2fa88e8c52a6846ca137c9d1b317c338d0ec1961bebe8445aa6343776f18f49f"}
Sep 19 05:55:53.637089 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.636722    2581 scope.go:117] "RemoveContainer" containerID="d7bfd2883e6ce7141bb11371a4e23c9f57ecfad03d6efe0df7a417c4cb1df201"
Sep 19 05:55:53.638108 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.638084    2581 generic.go:356] "Generic (PLEG): container finished" podID="66c82d38-f61e-4d88-a375-ac037694c6cb" containerID="6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247" exitCode=0
Sep 19 05:55:53.638177 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.638125    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"
Sep 19 05:55:53.638177 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.638138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf" event={"ID":"66c82d38-f61e-4d88-a375-ac037694c6cb","Type":"ContainerDied","Data":"6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247"}
Sep 19 05:55:53.638269 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.638179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf" event={"ID":"66c82d38-f61e-4d88-a375-ac037694c6cb","Type":"ContainerDied","Data":"d5bad36a0130034272d41ff0706c58e01baa6362666cd3644e4669506e35ecca"}
Sep 19 05:55:53.646619 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.646599    2581 scope.go:117] "RemoveContainer" containerID="6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247"
Sep 19 05:55:53.655133 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.655116    2581 scope.go:117] "RemoveContainer" containerID="6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247"
Sep 19 05:55:53.655409 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:55:53.655391    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247\": container with ID starting with 6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247 not found: ID does not exist" containerID="6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247"
Sep 19 05:55:53.655503 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.655416    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247"} err="failed to get container status \"6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247\": rpc error: code = NotFound desc = could not find container \"6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247\": container with ID starting with 6e53b2778aa19830b6e24b185ed70de91b3094e5310204b037c0b33168a84247 not found: ID does not exist"
Sep 19 05:55:53.663019 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.662996    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"]
Sep 19 05:55:53.670255 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.670234    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8128f3-54b8dd594-bkzbf"]
Sep 19 05:55:53.691838 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.687119    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"]
Sep 19 05:55:53.691838 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:53.688543    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8128f3-7796b95dd6-cqz4s"]
Sep 19 05:55:54.477050 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:54.477010    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66c82d38-f61e-4d88-a375-ac037694c6cb" path="/var/lib/kubelet/pods/66c82d38-f61e-4d88-a375-ac037694c6cb/volumes"
Sep 19 05:55:54.477520 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:54.477502    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4ebd4e-4648-4636-bc98-6aa7b44fab58" path="/var/lib/kubelet/pods/6b4ebd4e-4648-4636-bc98-6aa7b44fab58/volumes"
Sep 19 05:55:56.436473 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:56.436422    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:55:56.437665 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:56.437650    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 05:55:56.441544 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:56.441529    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:55:56.442377 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:55:56.442362    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 05:56:01.625532 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:01.625501    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:56:01.626002 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:01.625583    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:56:02.257808 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.257756    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq" podUID="9c25a189-d50f-4c7c-93ce-ebe3febce5be" containerName="main" containerID="cri-o://d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633" gracePeriod=45
Sep 19 05:56:02.515882 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.515808    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:56:02.558543 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558513    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tmp-dir\") pod \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") "
Sep 19 05:56:02.558733 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558612    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzgff\" (UniqueName: \"kubernetes.io/projected/9c25a189-d50f-4c7c-93ce-ebe3febce5be-kube-api-access-rzgff\") pod \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") "
Sep 19 05:56:02.558733 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-model-cache\") pod \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") "
Sep 19 05:56:02.558733 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tls-certs\") pod \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") "
Sep 19 05:56:02.558909 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-dshm\") pod \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") "
Sep 19 05:56:02.558909 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558759    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c25a189-d50f-4c7c-93ce-ebe3febce5be" (UID: "9c25a189-d50f-4c7c-93ce-ebe3febce5be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:02.558909 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-home\") pod \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\" (UID: \"9c25a189-d50f-4c7c-93ce-ebe3febce5be\") "
Sep 19 05:56:02.559078 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.558926    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c25a189-d50f-4c7c-93ce-ebe3febce5be" (UID: "9c25a189-d50f-4c7c-93ce-ebe3febce5be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:02.559173 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.559153    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:02.559239 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.559177    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:02.559314 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.559293    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-home" (OuterVolumeSpecName: "home") pod "9c25a189-d50f-4c7c-93ce-ebe3febce5be" (UID: "9c25a189-d50f-4c7c-93ce-ebe3febce5be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:02.561276 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.561252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c25a189-d50f-4c7c-93ce-ebe3febce5be" (UID: "9c25a189-d50f-4c7c-93ce-ebe3febce5be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:56:02.561408 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.561306    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-dshm" (OuterVolumeSpecName: "dshm") pod "9c25a189-d50f-4c7c-93ce-ebe3febce5be" (UID: "9c25a189-d50f-4c7c-93ce-ebe3febce5be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:02.561632 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.561422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c25a189-d50f-4c7c-93ce-ebe3febce5be-kube-api-access-rzgff" (OuterVolumeSpecName: "kube-api-access-rzgff") pod "9c25a189-d50f-4c7c-93ce-ebe3febce5be" (UID: "9c25a189-d50f-4c7c-93ce-ebe3febce5be"). InnerVolumeSpecName "kube-api-access-rzgff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:56:02.659903 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.659870    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzgff\" (UniqueName: \"kubernetes.io/projected/9c25a189-d50f-4c7c-93ce-ebe3febce5be-kube-api-access-rzgff\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:02.659903 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.659898    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c25a189-d50f-4c7c-93ce-ebe3febce5be-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:02.660305 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.659911    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:02.660305 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.659922    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c25a189-d50f-4c7c-93ce-ebe3febce5be-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:02.695207 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.695164    2581 generic.go:356] "Generic (PLEG): container finished" podID="9c25a189-d50f-4c7c-93ce-ebe3febce5be" containerID="d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633" exitCode=0
Sep 19 05:56:02.695368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.695293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq" event={"ID":"9c25a189-d50f-4c7c-93ce-ebe3febce5be","Type":"ContainerDied","Data":"d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633"}
Sep 19 05:56:02.695368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.695332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq" event={"ID":"9c25a189-d50f-4c7c-93ce-ebe3febce5be","Type":"ContainerDied","Data":"17deebdd08d94f4d29d81c92ae97adf8169aabd5cb1a4ae7ea9176be773971f8"}
Sep 19 05:56:02.695368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.695366    2581 scope.go:117] "RemoveContainer" containerID="d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633"
Sep 19 05:56:02.695590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.695572    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"
Sep 19 05:56:02.706691 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.706674    2581 scope.go:117] "RemoveContainer" containerID="d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633"
Sep 19 05:56:02.706976 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:02.706955    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633\": container with ID starting with d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633 not found: ID does not exist" containerID="d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633"
Sep 19 05:56:02.707047 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.706984    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633"} err="failed to get container status \"d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633\": rpc error: code = NotFound desc = could not find container \"d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633\": container with ID starting with d2048131404a5dd5fbc38e5f9b24a45eab1664a272bbf826ca45301c9e871633 not found: ID does not exist"
Sep 19 05:56:02.720803 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.720781    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"]
Sep 19 05:56:02.724906 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:02.724875    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-8128f3/autores-model-8128f3-kserve-58cc594f7f-fgfzq"]
Sep 19 05:56:04.477543 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:04.477502    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c25a189-d50f-4c7c-93ce-ebe3febce5be" path="/var/lib/kubelet/pods/9c25a189-d50f-4c7c-93ce-ebe3febce5be/volumes"
Sep 19 05:56:04.834999 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:04.834946    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"]
Sep 19 05:56:04.835244 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:04.835215    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" containerID="cri-o://2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077" gracePeriod=30
Sep 19 05:56:05.714374 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714342    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"]
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714732    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b4ebd4e-4648-4636-bc98-6aa7b44fab58" containerName="maas-api"
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714743    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4ebd4e-4648-4636-bc98-6aa7b44fab58" containerName="maas-api"
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714750    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c25a189-d50f-4c7c-93ce-ebe3febce5be" containerName="main"
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714755    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c25a189-d50f-4c7c-93ce-ebe3febce5be" containerName="main"
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714775    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c82d38-f61e-4d88-a375-ac037694c6cb" containerName="payload-pre-processing"
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714780    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c82d38-f61e-4d88-a375-ac037694c6cb" containerName="payload-pre-processing"
Sep 19 05:56:05.714829 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714834    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c25a189-d50f-4c7c-93ce-ebe3febce5be" containerName="main"
Sep 19 05:56:05.715157 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714844    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b4ebd4e-4648-4636-bc98-6aa7b44fab58" containerName="maas-api"
Sep 19 05:56:05.715157 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.714852    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c82d38-f61e-4d88-a375-ac037694c6cb" containerName="payload-pre-processing"
Sep 19 05:56:05.719511 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.719492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.723460 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.723437    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-875220-openshift-default-dockercfg-7pgcv\""
Sep 19 05:56:05.755717 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.755693    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"]
Sep 19 05:56:05.784305 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr2xk\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-kube-api-access-rr2xk\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753654e5-1c27-48bc-897d-de9180dea5de-istio-podinfo\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753654e5-1c27-48bc-897d-de9180dea5de-istiod-ca-cert\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-istio-token\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-credential-socket\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-envoy\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784774 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-socket\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784774 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-certs\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.784774 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.784615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-data\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.885872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.885835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-socket\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.885872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.885881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-certs\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.885901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-data\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.885940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr2xk\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-kube-api-access-rr2xk\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.885987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753654e5-1c27-48bc-897d-de9180dea5de-istio-podinfo\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753654e5-1c27-48bc-897d-de9180dea5de-istiod-ca-cert\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-istio-token\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-credential-socket\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-envoy\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886547 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-socket\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886547 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-data\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886547 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-credential-socket\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886719 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-certs\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.886913 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.886887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753654e5-1c27-48bc-897d-de9180dea5de-istiod-ca-cert\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.888647 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.888625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-envoy\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.888794 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.888771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753654e5-1c27-48bc-897d-de9180dea5de-istio-podinfo\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.898617 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.898591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-istio-token\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:05.898747 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:05.898713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr2xk\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-kube-api-access-rr2xk\") pod \"e2e-explicit-875220-openshift-default-8455794656-lnfxp\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") " pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:06.029984 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.029951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:06.305451 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.305424    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"]
Sep 19 05:56:06.307822 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:56:06.307796    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod753654e5_1c27_48bc_897d_de9180dea5de.slice/crio-4e40bf3c410a4e5d5c962a34201339fea97cb6d8e0c50e1145f74c84c898aaf4 WatchSource:0}: Error finding container 4e40bf3c410a4e5d5c962a34201339fea97cb6d8e0c50e1145f74c84c898aaf4: Status 404 returned error can't find the container with id 4e40bf3c410a4e5d5c962a34201339fea97cb6d8e0c50e1145f74c84c898aaf4
Sep 19 05:56:06.312672 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.312637    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:56:06.312769 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.312710    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:56:06.312769 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.312737    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:56:06.634364 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.634286    2581 patch_prober.go:28] interesting pod/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 05:56:06.634364 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.634347    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:56:06.722135 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.722102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" event={"ID":"753654e5-1c27-48bc-897d-de9180dea5de","Type":"ContainerStarted","Data":"492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"}
Sep 19 05:56:06.722511 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.722156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" event={"ID":"753654e5-1c27-48bc-897d-de9180dea5de","Type":"ContainerStarted","Data":"4e40bf3c410a4e5d5c962a34201339fea97cb6d8e0c50e1145f74c84c898aaf4"}
Sep 19 05:56:06.749648 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:06.749600    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podStartSLOduration=1.749584845 podStartE2EDuration="1.749584845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:06.747012184 +0000 UTC m=+912.357692175" watchObservedRunningTime="2026-09-19 05:56:06.749584845 +0000 UTC m=+912.360264808"
Sep 19 05:56:07.031097 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:07.031068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:07.032727 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:07.032701    2581 patch_prober.go:28] interesting pod/e2e-explicit-875220-openshift-default-8455794656-lnfxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 19 05:56:07.032845 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:07.032749    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 19 05:56:08.035143 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:08.035113    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:08.732468 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:08.732430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:08.733540 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:08.733520    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:10.115962 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.115937    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:56:10.230353 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w48f9\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-kube-api-access-w48f9\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230353 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230354    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-token\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230389    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istiod-ca-cert\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230409    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-socket\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230438    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-envoy\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230462    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-credential-socket\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230502    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-data\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-certs\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230886 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230589    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-podinfo\") pod \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\" (UID: \"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1\") "
Sep 19 05:56:10.230886 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230696    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:10.231001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230920    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.231001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230935    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:10.231001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.230967    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:56:10.231204 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.231003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:10.231429 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.231284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-data" (OuterVolumeSpecName: "istio-data") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:10.232946 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.232839    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-token" (OuterVolumeSpecName: "istio-token") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:56:10.233221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.233201    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 05:56:10.233369 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.233352    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-kube-api-access-w48f9" (OuterVolumeSpecName: "kube-api-access-w48f9") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "kube-api-access-w48f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:56:10.233420 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.233363    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" (UID: "e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:56:10.332150 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332110    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-credential-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332150 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332144    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-data\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332162    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-workload-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332175    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-podinfo\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332189    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w48f9\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-kube-api-access-w48f9\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332202    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332214    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istiod-ca-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.332389 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.332225    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1-istio-envoy\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:10.744501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.744403    2581 generic.go:356] "Generic (PLEG): container finished" podID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerID="2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077" exitCode=0
Sep 19 05:56:10.744501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.744449    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"
Sep 19 05:56:10.744501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.744496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" event={"ID":"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1","Type":"ContainerDied","Data":"2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077"}
Sep 19 05:56:10.744795 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.744541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf" event={"ID":"e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1","Type":"ContainerDied","Data":"83a10e6d4d906669699beec1dc99e4356c449afb3c67bffcb824b408cef87679"}
Sep 19 05:56:10.744795 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.744571    2581 scope.go:117] "RemoveContainer" containerID="2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077"
Sep 19 05:56:10.753632 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.753616    2581 scope.go:117] "RemoveContainer" containerID="85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"
Sep 19 05:56:10.762206 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.762186    2581 scope.go:117] "RemoveContainer" containerID="2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077"
Sep 19 05:56:10.762530 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:10.762509    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077\": container with ID starting with 2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077 not found: ID does not exist" containerID="2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077"
Sep 19 05:56:10.762589 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.762543    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077"} err="failed to get container status \"2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077\": rpc error: code = NotFound desc = could not find container \"2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077\": container with ID starting with 2277902db1592b29f4d7e9f7851e135d847fb902aeb83a7955f9ac9533998077 not found: ID does not exist"
Sep 19 05:56:10.762589 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.762561    2581 scope.go:117] "RemoveContainer" containerID="85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"
Sep 19 05:56:10.762825 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:10.762808    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a\": container with ID starting with 85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a not found: ID does not exist" containerID="85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"
Sep 19 05:56:10.762861 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.762830    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a"} err="failed to get container status \"85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a\": rpc error: code = NotFound desc = could not find container \"85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a\": container with ID starting with 85b7ac259b2f4d520d031f3daa1078f5e1824ecb7ab2cf2d93d47ee819f7832a not found: ID does not exist"
Sep 19 05:56:10.763887 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.763867    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"]
Sep 19 05:56:10.775230 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:10.775210    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-8128f3-openshift-default-6dff9b8bb6-j2btf"]
Sep 19 05:56:12.477450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:12.477413    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" path="/var/lib/kubelet/pods/e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1/volumes"
Sep 19 05:56:12.758980 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:12.758905    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:56:13.392421 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.392375    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"]
Sep 19 05:56:13.393029 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.392993    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy"
Sep 19 05:56:13.393029 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.393021    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy"
Sep 19 05:56:13.393241 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.393195    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy"
Sep 19 05:56:13.393241 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.393210    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy"
Sep 19 05:56:13.398361 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.398341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.401092 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.401069    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-875220-serving-cert\""
Sep 19 05:56:13.409574 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.409544    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"]
Sep 19 05:56:13.464754 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.464713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a75103d8-19df-4673-ac39-5abd13b79b18-openshift-service-ca\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.464754 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.464754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-metrics-certs\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.464975 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.464857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.464975 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.464909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kphdx\" (UniqueName: \"kubernetes.io/projected/a75103d8-19df-4673-ac39-5abd13b79b18-kube-api-access-kphdx\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.565461 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.565419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.565868 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.565480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kphdx\" (UniqueName: \"kubernetes.io/projected/a75103d8-19df-4673-ac39-5abd13b79b18-kube-api-access-kphdx\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.565868 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.565527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a75103d8-19df-4673-ac39-5abd13b79b18-openshift-service-ca\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.565868 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.565545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-metrics-certs\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.565868 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:13.565575    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-875220-serving-cert: secret "maas-api-e2e-explicit-875220-serving-cert" not found
Sep 19 05:56:13.565868 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:13.565669    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls podName:a75103d8-19df-4673-ac39-5abd13b79b18 nodeName:}" failed. No retries permitted until 2026-09-19 05:56:14.065645434 +0000 UTC m=+919.676325409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls") pod "maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" (UID: "a75103d8-19df-4673-ac39-5abd13b79b18") : secret "maas-api-e2e-explicit-875220-serving-cert" not found
Sep 19 05:56:13.566226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.566211    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a75103d8-19df-4673-ac39-5abd13b79b18-openshift-service-ca\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.568101 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.568074    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-metrics-certs\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:13.577547 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:13.577521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kphdx\" (UniqueName: \"kubernetes.io/projected/a75103d8-19df-4673-ac39-5abd13b79b18-kube-api-access-kphdx\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:14.070404 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.070363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:14.072881 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.072853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls\") pod \"maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:14.166961 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.166928    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"]
Sep 19 05:56:14.167367 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.167353    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy"
Sep 19 05:56:14.167420 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.167369    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e55d0bcf-5ee8-4f99-a55b-736b1da5bbc1" containerName="istio-proxy"
Sep 19 05:56:14.170417 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.170399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.173243 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.173223    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-875220-dockercfg-6xngb\""
Sep 19 05:56:14.173368 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.173263    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-875220\""
Sep 19 05:56:14.184782 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.184759    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"]
Sep 19 05:56:14.271631 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.271595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt44d\" (UniqueName: \"kubernetes.io/projected/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-kube-api-access-pt44d\") pod \"payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.271817 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.271722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.312796 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.312764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:14.364383 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.364349    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"]
Sep 19 05:56:14.364699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.364614    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc" podUID="033bec5d-16e9-4f56-b2b3-e2f63072b00a" containerName="payload-processing" containerID="cri-o://d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56" gracePeriod=30
Sep 19 05:56:14.372915 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.372524    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"]
Sep 19 05:56:14.372915 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.372720    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck" podUID="095676c6-f7ab-46a8-9d17-81cd86063d0b" containerName="payload-pre-processing" containerID="cri-o://a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7" gracePeriod=30
Sep 19 05:56:14.373875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.373380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.373875 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.373506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt44d\" (UniqueName: \"kubernetes.io/projected/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-kube-api-access-pt44d\") pod \"payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.375201 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.374583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.389090 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.388663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt44d\" (UniqueName: \"kubernetes.io/projected/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-kube-api-access-pt44d\") pod \"payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.480793 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.480767    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:14.600398 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.600271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"]
Sep 19 05:56:14.604623 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:56:14.604597    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75103d8_19df_4673_ac39_5abd13b79b18.slice/crio-947bf3f5f6d31b1e8fc5c58205f11df89ceeba8b0a439c18ad64598b003d216f WatchSource:0}: Error finding container 947bf3f5f6d31b1e8fc5c58205f11df89ceeba8b0a439c18ad64598b003d216f: Status 404 returned error can't find the container with id 947bf3f5f6d31b1e8fc5c58205f11df89ceeba8b0a439c18ad64598b003d216f
Sep 19 05:56:14.667116 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.667094    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:56:14.669114 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.669089    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:56:14.761120 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.761097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"]
Sep 19 05:56:14.763381 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:56:14.763354    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535b2d4b_b2eb_4ace_b3cf_cc99c81e62ab.slice/crio-13a7d4df50224fca0839516cb02588e864b414035303c56b86e8ff60842f008b WatchSource:0}: Error finding container 13a7d4df50224fca0839516cb02588e864b414035303c56b86e8ff60842f008b: Status 404 returned error can't find the container with id 13a7d4df50224fca0839516cb02588e864b414035303c56b86e8ff60842f008b
Sep 19 05:56:14.767642 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.767619    2581 generic.go:356] "Generic (PLEG): container finished" podID="033bec5d-16e9-4f56-b2b3-e2f63072b00a" containerID="d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56" exitCode=0
Sep 19 05:56:14.767716 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.767658    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"
Sep 19 05:56:14.767783 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.767702    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc" event={"ID":"033bec5d-16e9-4f56-b2b3-e2f63072b00a","Type":"ContainerDied","Data":"d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56"}
Sep 19 05:56:14.767783 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.767763    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc" event={"ID":"033bec5d-16e9-4f56-b2b3-e2f63072b00a","Type":"ContainerDied","Data":"93d1acd63d0be1ea2baa8fc9d5b9df223bdfc3992cd1340cfc2c04597ccd4f30"}
Sep 19 05:56:14.767899 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.767794    2581 scope.go:117] "RemoveContainer" containerID="d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56"
Sep 19 05:56:14.768968 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.768951    2581 generic.go:356] "Generic (PLEG): container finished" podID="095676c6-f7ab-46a8-9d17-81cd86063d0b" containerID="a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7" exitCode=0
Sep 19 05:56:14.769068 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.769006    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"
Sep 19 05:56:14.769068 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.769022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck" event={"ID":"095676c6-f7ab-46a8-9d17-81cd86063d0b","Type":"ContainerDied","Data":"a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7"}
Sep 19 05:56:14.769184 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.769082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck" event={"ID":"095676c6-f7ab-46a8-9d17-81cd86063d0b","Type":"ContainerDied","Data":"72f3419b70caa0712cb3376a55b73ece7f34d940f39035b279be20abee4ee80f"}
Sep 19 05:56:14.770449 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.770294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" event={"ID":"a75103d8-19df-4673-ac39-5abd13b79b18","Type":"ContainerStarted","Data":"947bf3f5f6d31b1e8fc5c58205f11df89ceeba8b0a439c18ad64598b003d216f"}
Sep 19 05:56:14.777728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.777002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095676c6-f7ab-46a8-9d17-81cd86063d0b-plugins-config-volume\") pod \"095676c6-f7ab-46a8-9d17-81cd86063d0b\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") "
Sep 19 05:56:14.777728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.777088    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn447\" (UniqueName: \"kubernetes.io/projected/095676c6-f7ab-46a8-9d17-81cd86063d0b-kube-api-access-fn447\") pod \"095676c6-f7ab-46a8-9d17-81cd86063d0b\" (UID: \"095676c6-f7ab-46a8-9d17-81cd86063d0b\") "
Sep 19 05:56:14.777728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.777120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/033bec5d-16e9-4f56-b2b3-e2f63072b00a-plugins-config-volume\") pod \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") "
Sep 19 05:56:14.777728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.777176    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jshzc\" (UniqueName: \"kubernetes.io/projected/033bec5d-16e9-4f56-b2b3-e2f63072b00a-kube-api-access-jshzc\") pod \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\" (UID: \"033bec5d-16e9-4f56-b2b3-e2f63072b00a\") "
Sep 19 05:56:14.778265 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.778239    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/095676c6-f7ab-46a8-9d17-81cd86063d0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "095676c6-f7ab-46a8-9d17-81cd86063d0b" (UID: "095676c6-f7ab-46a8-9d17-81cd86063d0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:56:14.778623 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.778595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/033bec5d-16e9-4f56-b2b3-e2f63072b00a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "033bec5d-16e9-4f56-b2b3-e2f63072b00a" (UID: "033bec5d-16e9-4f56-b2b3-e2f63072b00a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:56:14.779873 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.779841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/095676c6-f7ab-46a8-9d17-81cd86063d0b-kube-api-access-fn447" (OuterVolumeSpecName: "kube-api-access-fn447") pod "095676c6-f7ab-46a8-9d17-81cd86063d0b" (UID: "095676c6-f7ab-46a8-9d17-81cd86063d0b"). InnerVolumeSpecName "kube-api-access-fn447". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:56:14.780064 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.779985    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033bec5d-16e9-4f56-b2b3-e2f63072b00a-kube-api-access-jshzc" (OuterVolumeSpecName: "kube-api-access-jshzc") pod "033bec5d-16e9-4f56-b2b3-e2f63072b00a" (UID: "033bec5d-16e9-4f56-b2b3-e2f63072b00a"). InnerVolumeSpecName "kube-api-access-jshzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:56:14.780183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.780154    2581 scope.go:117] "RemoveContainer" containerID="d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56"
Sep 19 05:56:14.780494 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:14.780477    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56\": container with ID starting with d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56 not found: ID does not exist" containerID="d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56"
Sep 19 05:56:14.780559 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.780501    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56"} err="failed to get container status \"d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56\": rpc error: code = NotFound desc = could not find container \"d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56\": container with ID starting with d49ca31b8217037910c11d336e42da299597064519ff415be58ede48a48bde56 not found: ID does not exist"
Sep 19 05:56:14.780559 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.780516    2581 scope.go:117] "RemoveContainer" containerID="a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7"
Sep 19 05:56:14.789431 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.789407    2581 scope.go:117] "RemoveContainer" containerID="a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7"
Sep 19 05:56:14.789721 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:56:14.789693    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7\": container with ID starting with a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7 not found: ID does not exist" containerID="a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7"
Sep 19 05:56:14.789815 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.789728    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7"} err="failed to get container status \"a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7\": rpc error: code = NotFound desc = could not find container \"a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7\": container with ID starting with a4f96707adb29386eca7cf23a1e782c9cc92940dee07f2fbe528dbf8e5a411b7 not found: ID does not exist"
Sep 19 05:56:14.878575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.878504    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/095676c6-f7ab-46a8-9d17-81cd86063d0b-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:14.878575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.878528    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn447\" (UniqueName: \"kubernetes.io/projected/095676c6-f7ab-46a8-9d17-81cd86063d0b-kube-api-access-fn447\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:14.878575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.878539    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/033bec5d-16e9-4f56-b2b3-e2f63072b00a-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:14.878575 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:14.878547    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jshzc\" (UniqueName: \"kubernetes.io/projected/033bec5d-16e9-4f56-b2b3-e2f63072b00a-kube-api-access-jshzc\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:56:15.095675 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.095640    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"]
Sep 19 05:56:15.104051 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.102377    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-39e34e62-6dc5c7bfb5-t9vpc"]
Sep 19 05:56:15.116687 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.116660    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"]
Sep 19 05:56:15.121621 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.121595    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-39e34e62-5f95b9dff4-r2kck"]
Sep 19 05:56:15.778075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.778045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" event={"ID":"a75103d8-19df-4673-ac39-5abd13b79b18","Type":"ContainerStarted","Data":"826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197"}
Sep 19 05:56:15.778515 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.778171    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:15.779674 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.779655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6" event={"ID":"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab","Type":"ContainerStarted","Data":"3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96"}
Sep 19 05:56:15.779748 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.779693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6" event={"ID":"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab","Type":"ContainerStarted","Data":"13a7d4df50224fca0839516cb02588e864b414035303c56b86e8ff60842f008b"}
Sep 19 05:56:15.779748 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.779713    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:15.820560 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.820509    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" podStartSLOduration=2.295865492 podStartE2EDuration="2.820491548s" totalImagesPullingTime="524.626056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:15.819929284 +0000 UTC m=+921.430609274" watchObservedRunningTime="2026-09-19 05:56:15.820491548 +0000 UTC m=+921.431171511"
Sep 19 05:56:15.859551 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:15.859466    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6" podStartSLOduration=1.859453215 podStartE2EDuration="1.859453215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:56:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:15.85725141 +0000 UTC m=+921.467931400" watchObservedRunningTime="2026-09-19 05:56:15.859453215 +0000 UTC m=+921.470133181"
Sep 19 05:56:16.478423 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:16.478390    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="033bec5d-16e9-4f56-b2b3-e2f63072b00a" path="/var/lib/kubelet/pods/033bec5d-16e9-4f56-b2b3-e2f63072b00a/volumes"
Sep 19 05:56:16.478783 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:16.478769    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="095676c6-f7ab-46a8-9d17-81cd86063d0b" path="/var/lib/kubelet/pods/095676c6-f7ab-46a8-9d17-81cd86063d0b/volumes"
Sep 19 05:56:21.790224 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:21.790195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:56:26.786849 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:26.786820    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:56:32.664963 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.664925    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"]
Sep 19 05:56:32.665508 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.665480    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="033bec5d-16e9-4f56-b2b3-e2f63072b00a" containerName="payload-processing"
Sep 19 05:56:32.665508 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.665508    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="033bec5d-16e9-4f56-b2b3-e2f63072b00a" containerName="payload-processing"
Sep 19 05:56:32.665686 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.665526    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="095676c6-f7ab-46a8-9d17-81cd86063d0b" containerName="payload-pre-processing"
Sep 19 05:56:32.665686 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.665535    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="095676c6-f7ab-46a8-9d17-81cd86063d0b" containerName="payload-pre-processing"
Sep 19 05:56:32.665686 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.665646    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="095676c6-f7ab-46a8-9d17-81cd86063d0b" containerName="payload-pre-processing"
Sep 19 05:56:32.665686 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.665661    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="033bec5d-16e9-4f56-b2b3-e2f63072b00a" containerName="payload-processing"
Sep 19 05:56:32.669831 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.669814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.674245 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.673210    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"premium-w4-3182b5-kserve-self-signed-certs\""
Sep 19 05:56:32.685893 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.685866    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"]
Sep 19 05:56:32.846481 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.846444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tls-certs\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.846680 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.846493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-home\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.846680 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.846518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-dshm\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.846680 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.846644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdfq2\" (UniqueName: \"kubernetes.io/projected/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-kube-api-access-jdfq2\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.846822 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.846701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tmp-dir\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.846822 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.846741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-model-cache\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948136 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdfq2\" (UniqueName: \"kubernetes.io/projected/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-kube-api-access-jdfq2\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948136 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tmp-dir\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-model-cache\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tls-certs\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-home\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-dshm\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948542 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tmp-dir\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948611 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-model-cache\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.948657 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.948634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-home\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.950783 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.950759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-dshm\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.950920 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.950904    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tls-certs\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.956550 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.956531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdfq2\" (UniqueName: \"kubernetes.io/projected/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-kube-api-access-jdfq2\") pod \"premium-w4-3182b5-kserve-776fb96d85-7n5vz\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") " pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:32.986799 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:32.986771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:33.263558 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:33.263524    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"]
Sep 19 05:56:33.265519 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:56:33.265477    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ea96e3c_a4dc_44b0_9d0d_ecae36f27e13.slice/crio-bfd7d79271a877ac4955d5b97c53964a480ee8fe3d103e56860976ee3c08c888 WatchSource:0}: Error finding container bfd7d79271a877ac4955d5b97c53964a480ee8fe3d103e56860976ee3c08c888: Status 404 returned error can't find the container with id bfd7d79271a877ac4955d5b97c53964a480ee8fe3d103e56860976ee3c08c888
Sep 19 05:56:33.878173 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:33.878137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz" event={"ID":"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13","Type":"ContainerStarted","Data":"01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b"}
Sep 19 05:56:33.878173 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:33.878178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz" event={"ID":"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13","Type":"ContainerStarted","Data":"bfd7d79271a877ac4955d5b97c53964a480ee8fe3d103e56860976ee3c08c888"}
Sep 19 05:56:33.899499 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:33.899431    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz" podStartSLOduration=1.899406919 podStartE2EDuration="1.899406919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:33.897202154 +0000 UTC m=+939.507882183" watchObservedRunningTime="2026-09-19 05:56:33.899406919 +0000 UTC m=+939.510086885"
Sep 19 05:56:34.345999 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.345958    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:56:34.885569 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.885533    2581 generic.go:356] "Generic (PLEG): container finished" podID="753654e5-1c27-48bc-897d-de9180dea5de" containerID="492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854" exitCode=0
Sep 19 05:56:34.885961 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.885576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" event={"ID":"753654e5-1c27-48bc-897d-de9180dea5de","Type":"ContainerDied","Data":"492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"}
Sep 19 05:56:34.886059 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.886019    2581 scope.go:117] "RemoveContainer" containerID="492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"
Sep 19 05:56:34.887185 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.887155    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:56:34.887280 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.887238    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:56:34.887280 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:34.887271    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:56:35.896169 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:35.896133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" event={"ID":"753654e5-1c27-48bc-897d-de9180dea5de","Type":"ContainerStarted","Data":"aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89"}
Sep 19 05:56:36.030972 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:36.030937    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:36.032251 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:36.032226    2581 patch_prober.go:28] interesting pod/e2e-explicit-875220-openshift-default-8455794656-lnfxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 19 05:56:36.032312 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:36.032292    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 19 05:56:37.030425 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:37.030388    2581 patch_prober.go:28] interesting pod/e2e-explicit-875220-openshift-default-8455794656-lnfxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 19 05:56:37.030887 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:37.030451    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 19 05:56:38.030757 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:38.030719    2581 patch_prober.go:28] interesting pod/e2e-explicit-875220-openshift-default-8455794656-lnfxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 19 05:56:38.031167 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:38.030777    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 19 05:56:38.733300 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:38.733253    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:39.031001 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:39.030964    2581 patch_prober.go:28] interesting pod/e2e-explicit-875220-openshift-default-8455794656-lnfxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 19 05:56:39.031486 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:39.031023    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 19 05:56:40.030764 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:40.030728    2581 patch_prober.go:28] interesting pod/e2e-explicit-875220-openshift-default-8455794656-lnfxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 19 05:56:40.030943 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:40.030787    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 19 05:56:41.204885 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:41.204849    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:41.206161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:41.206135    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:56:42.987549 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:42.987521    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:42.987942 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:42.987562    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:43.001226 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:43.001197    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:43.958152 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:43.958124    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 05:56:54.259044 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:56:54.259007    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:57:01.144586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.144552    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"]
Sep 19 05:57:01.149818 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.149797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.153876 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.153840    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c85316\"/\"openshift-service-ca.crt\""
Sep 19 05:57:01.153876 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.153857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c85316\"/\"default-dockercfg-kxx79\""
Sep 19 05:57:01.153876 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.153840    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c85316\"/\"test-model-c85316-kserve-self-signed-certs\""
Sep 19 05:57:01.153876 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.153871    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c85316\"/\"kube-root-ca.crt\""
Sep 19 05:57:01.160857 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.160830    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"]
Sep 19 05:57:01.224904 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.224868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-model-cache\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.225096 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.224934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tls-certs\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.225096 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.224976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-home\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.225096 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.225066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tmp-dir\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.225096 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.225089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwxvg\" (UniqueName: \"kubernetes.io/projected/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-kube-api-access-mwxvg\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.225271 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.225169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-dshm\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.325727 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.325684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tls-certs\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.325727 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.325730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-home\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.325973 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.325771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tmp-dir\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.325973 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.325788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwxvg\" (UniqueName: \"kubernetes.io/projected/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-kube-api-access-mwxvg\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.325973 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.325813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-dshm\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.325973 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.325881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-model-cache\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.326286 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.326261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tmp-dir\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.326286 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.326277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-model-cache\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.326648 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.326623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-home\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.328536 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.328512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-dshm\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.328628 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.328595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tls-certs\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.334714 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.334688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwxvg\" (UniqueName: \"kubernetes.io/projected/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-kube-api-access-mwxvg\") pod \"test-model-c85316-kserve-9d84bc597-xxmlm\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") " pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.462161 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.462081    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:01.736743 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:01.736719    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"]
Sep 19 05:57:01.738977 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:57:01.738936    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f5ba7af_9e13_4f33_81a6_0bbe252c21e6.slice/crio-a62d15d2a3af24a3274dae559f6e43d1645d1b63dab73596aed07ce6dd105a42 WatchSource:0}: Error finding container a62d15d2a3af24a3274dae559f6e43d1645d1b63dab73596aed07ce6dd105a42: Status 404 returned error can't find the container with id a62d15d2a3af24a3274dae559f6e43d1645d1b63dab73596aed07ce6dd105a42
Sep 19 05:57:02.038221 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:02.038181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm" event={"ID":"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6","Type":"ContainerStarted","Data":"7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a"}
Sep 19 05:57:02.038410 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:02.038230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm" event={"ID":"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6","Type":"ContainerStarted","Data":"a62d15d2a3af24a3274dae559f6e43d1645d1b63dab73596aed07ce6dd105a42"}
Sep 19 05:57:02.056340 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:02.056293    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm" podStartSLOduration=1.056278868 podStartE2EDuration="1.056278868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:57:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:57:02.053628247 +0000 UTC m=+967.664308229" watchObservedRunningTime="2026-09-19 05:57:02.056278868 +0000 UTC m=+967.666958876"
Sep 19 05:57:11.462525 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:11.462489    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:11.462901 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:11.462551    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:11.475072 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:11.475043    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:12.100606 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:12.100571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 05:57:24.251176 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:24.251142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:57:45.056403 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:45.056365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:57:54.695902 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:54.695866    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"]
Sep 19 05:57:54.696814 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:54.696775    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" podUID="a75103d8-19df-4673-ac39-5abd13b79b18" containerName="maas-api" containerID="cri-o://826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197" gracePeriod=30
Sep 19 05:57:54.759543 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:54.759506    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"]
Sep 19 05:57:54.759781 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:54.759754    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6" podUID="535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" containerName="payload-pre-processing" containerID="cri-o://3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96" gracePeriod=30
Sep 19 05:57:55.038648 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.038624    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:57:55.041606 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.041582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:57:55.152521 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152492    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-plugins-config-volume\") pod \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") "
Sep 19 05:57:55.152698 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt44d\" (UniqueName: \"kubernetes.io/projected/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-kube-api-access-pt44d\") pod \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\" (UID: \"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab\") "
Sep 19 05:57:55.152698 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152587    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kphdx\" (UniqueName: \"kubernetes.io/projected/a75103d8-19df-4673-ac39-5abd13b79b18-kube-api-access-kphdx\") pod \"a75103d8-19df-4673-ac39-5abd13b79b18\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") "
Sep 19 05:57:55.152698 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152617    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a75103d8-19df-4673-ac39-5abd13b79b18-openshift-service-ca\") pod \"a75103d8-19df-4673-ac39-5abd13b79b18\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") "
Sep 19 05:57:55.152698 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152661    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls\") pod \"a75103d8-19df-4673-ac39-5abd13b79b18\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") "
Sep 19 05:57:55.152929 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-metrics-certs\") pod \"a75103d8-19df-4673-ac39-5abd13b79b18\" (UID: \"a75103d8-19df-4673-ac39-5abd13b79b18\") "
Sep 19 05:57:55.153063 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.152938    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" (UID: "535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:57:55.153133 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.153084    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a75103d8-19df-4673-ac39-5abd13b79b18-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a75103d8-19df-4673-ac39-5abd13b79b18" (UID: "a75103d8-19df-4673-ac39-5abd13b79b18"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:57:55.153641 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.153594    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:57:55.153641 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.153626    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a75103d8-19df-4673-ac39-5abd13b79b18-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:57:55.155127 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.155095    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a75103d8-19df-4673-ac39-5abd13b79b18" (UID: "a75103d8-19df-4673-ac39-5abd13b79b18"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:57:55.155248 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.155178    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a75103d8-19df-4673-ac39-5abd13b79b18" (UID: "a75103d8-19df-4673-ac39-5abd13b79b18"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:57:55.155460 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.155438    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75103d8-19df-4673-ac39-5abd13b79b18-kube-api-access-kphdx" (OuterVolumeSpecName: "kube-api-access-kphdx") pod "a75103d8-19df-4673-ac39-5abd13b79b18" (UID: "a75103d8-19df-4673-ac39-5abd13b79b18"). InnerVolumeSpecName "kube-api-access-kphdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:57:55.155561 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.155475    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-kube-api-access-pt44d" (OuterVolumeSpecName: "kube-api-access-pt44d") pod "535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" (UID: "535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab"). InnerVolumeSpecName "kube-api-access-pt44d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:57:55.254586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.254511    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt44d\" (UniqueName: \"kubernetes.io/projected/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab-kube-api-access-pt44d\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:57:55.254586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.254541    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kphdx\" (UniqueName: \"kubernetes.io/projected/a75103d8-19df-4673-ac39-5abd13b79b18-kube-api-access-kphdx\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:57:55.254586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.254550    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:57:55.254586 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.254561    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a75103d8-19df-4673-ac39-5abd13b79b18-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:57:55.313326 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.313299    2581 generic.go:356] "Generic (PLEG): container finished" podID="a75103d8-19df-4673-ac39-5abd13b79b18" containerID="826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197" exitCode=0
Sep 19 05:57:55.313502 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.313344    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"
Sep 19 05:57:55.313502 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.313382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" event={"ID":"a75103d8-19df-4673-ac39-5abd13b79b18","Type":"ContainerDied","Data":"826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197"}
Sep 19 05:57:55.313502 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.313460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp" event={"ID":"a75103d8-19df-4673-ac39-5abd13b79b18","Type":"ContainerDied","Data":"947bf3f5f6d31b1e8fc5c58205f11df89ceeba8b0a439c18ad64598b003d216f"}
Sep 19 05:57:55.313502 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.313484    2581 scope.go:117] "RemoveContainer" containerID="826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197"
Sep 19 05:57:55.314881 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.314859    2581 generic.go:356] "Generic (PLEG): container finished" podID="535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" containerID="3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96" exitCode=0
Sep 19 05:57:55.314965 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.314896    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"
Sep 19 05:57:55.314965 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.314940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6" event={"ID":"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab","Type":"ContainerDied","Data":"3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96"}
Sep 19 05:57:55.315112 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.314987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6" event={"ID":"535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab","Type":"ContainerDied","Data":"13a7d4df50224fca0839516cb02588e864b414035303c56b86e8ff60842f008b"}
Sep 19 05:57:55.324194 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.324177    2581 scope.go:117] "RemoveContainer" containerID="826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197"
Sep 19 05:57:55.324481 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:57:55.324459    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197\": container with ID starting with 826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197 not found: ID does not exist" containerID="826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197"
Sep 19 05:57:55.324564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.324488    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197"} err="failed to get container status \"826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197\": rpc error: code = NotFound desc = could not find container \"826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197\": container with ID starting with 826e93a9434ec13ac19cae2a671dfd8df4425956705068596f42f9597d5cb197 not found: ID does not exist"
Sep 19 05:57:55.324564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.324511    2581 scope.go:117] "RemoveContainer" containerID="3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96"
Sep 19 05:57:55.332417 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.332401    2581 scope.go:117] "RemoveContainer" containerID="3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96"
Sep 19 05:57:55.332676 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:57:55.332660    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96\": container with ID starting with 3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96 not found: ID does not exist" containerID="3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96"
Sep 19 05:57:55.332716 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.332684    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96"} err="failed to get container status \"3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96\": rpc error: code = NotFound desc = could not find container \"3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96\": container with ID starting with 3da9e16a432f385321126064f83d1a1b16e7f672a6bac0f7ac5be5631d0b1a96 not found: ID does not exist"
Sep 19 05:57:55.336853 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.336832    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"]
Sep 19 05:57:55.340455 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.340431    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-875220-7d94cfff8c-z2pdp"]
Sep 19 05:57:55.352964 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.352944    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"]
Sep 19 05:57:55.356775 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:55.356741    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-875220-98bf89f4b-v62s6"]
Sep 19 05:57:56.477522 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:56.477492    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" path="/var/lib/kubelet/pods/535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab/volumes"
Sep 19 05:57:56.477877 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:57:56.477848    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a75103d8-19df-4673-ac39-5abd13b79b18" path="/var/lib/kubelet/pods/a75103d8-19df-4673-ac39-5abd13b79b18/volumes"
Sep 19 05:58:02.643950 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.643915    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"]
Sep 19 05:58:02.644526 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.644198    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy" containerID="cri-o://aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89" gracePeriod=30
Sep 19 05:58:02.745798 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.745765    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"]
Sep 19 05:58:02.746165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.746151    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a75103d8-19df-4673-ac39-5abd13b79b18" containerName="maas-api"
Sep 19 05:58:02.746223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.746166    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a75103d8-19df-4673-ac39-5abd13b79b18" containerName="maas-api"
Sep 19 05:58:02.746223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.746177    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" containerName="payload-pre-processing"
Sep 19 05:58:02.746223 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.746183    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" containerName="payload-pre-processing"
Sep 19 05:58:02.746316 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.746249    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="535b2d4b-b2eb-4ace-b3cf-cc99c81e62ab" containerName="payload-pre-processing"
Sep 19 05:58:02.746316 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.746257    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a75103d8-19df-4673-ac39-5abd13b79b18" containerName="maas-api"
Sep 19 05:58:02.750928 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.750909    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:02.758321 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.758298    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-aec2317b\""
Sep 19 05:58:02.758427 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.758298    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-aec2317b-dockercfg-d75hh\""
Sep 19 05:58:02.770453 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.770430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"]
Sep 19 05:58:02.824015 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.823984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fpvz\" (UniqueName: \"kubernetes.io/projected/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-kube-api-access-9fpvz\") pod \"payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:02.824298 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.824070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:02.925337 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.925251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fpvz\" (UniqueName: \"kubernetes.io/projected/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-kube-api-access-9fpvz\") pod \"payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:02.925337 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.925314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:02.925970 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.925952    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:02.933590 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:02.933563    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fpvz\" (UniqueName: \"kubernetes.io/projected/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-kube-api-access-9fpvz\") pod \"payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:03.062074 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.062042    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:03.338701 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.338666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"]
Sep 19 05:58:03.342071 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:58:03.342045    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b917e5_a12f_41bd_b68c_7f7435bc7ff0.slice/crio-b1c3d9e4de53b35ef36eab6d63b1f4f8524b4fc1946b82ebe5333994093e94e7 WatchSource:0}: Error finding container b1c3d9e4de53b35ef36eab6d63b1f4f8524b4fc1946b82ebe5333994093e94e7: Status 404 returned error can't find the container with id b1c3d9e4de53b35ef36eab6d63b1f4f8524b4fc1946b82ebe5333994093e94e7
Sep 19 05:58:03.360421 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.360393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc" event={"ID":"72b917e5-a12f-41bd-b68c-7f7435bc7ff0","Type":"ContainerStarted","Data":"b1c3d9e4de53b35ef36eab6d63b1f4f8524b4fc1946b82ebe5333994093e94e7"}
Sep 19 05:58:03.419992 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.419958    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"]
Sep 19 05:58:03.423748 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.423726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.426608 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.426581    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-2d8e5f-openshift-default-dockercfg-lhx89\""
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-envoy\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-credential-socket\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-istio-token\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f20ac1-3b85-4c4f-824d-d52972474316-istiod-ca-cert\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-certs\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-data\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f20ac1-3b85-4c4f-824d-d52972474316-istio-podinfo\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-socket\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.431581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.431441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9szh7\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-kube-api-access-9szh7\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.436672 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.436651    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"]
Sep 19 05:58:03.532948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.532907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-data\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.532948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.532952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f20ac1-3b85-4c4f-824d-d52972474316-istio-podinfo\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.532972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-socket\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9szh7\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-kube-api-access-9szh7\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-envoy\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-credential-socket\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-istio-token\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f20ac1-3b85-4c4f-824d-d52972474316-istiod-ca-cert\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533214 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-certs\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533592 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533387    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-data\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533592 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-certs\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533592 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-socket\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.533737 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.533628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-credential-socket\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.534461 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.534436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f20ac1-3b85-4c4f-824d-d52972474316-istiod-ca-cert\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.535828 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.535803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f20ac1-3b85-4c4f-824d-d52972474316-istio-podinfo\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.535928 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.535877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-envoy\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.540966 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.540946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-istio-token\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.541450 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.541428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9szh7\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-kube-api-access-9szh7\") pod \"e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") " pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:03.736088 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:03.735985    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:04.033410 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.033381    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"]
Sep 19 05:58:04.034273 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:58:04.034248    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f20ac1_3b85_4c4f_824d_d52972474316.slice/crio-3de61f892ed6524860902bc0a2f188f89888cda5e4889fb5b05748c5f06d6756 WatchSource:0}: Error finding container 3de61f892ed6524860902bc0a2f188f89888cda5e4889fb5b05748c5f06d6756: Status 404 returned error can't find the container with id 3de61f892ed6524860902bc0a2f188f89888cda5e4889fb5b05748c5f06d6756
Sep 19 05:58:04.037043 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.037001    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:58:04.037119 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.037092    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:58:04.037163 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.037124    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:58:04.368841 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.368736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc" event={"ID":"72b917e5-a12f-41bd-b68c-7f7435bc7ff0","Type":"ContainerStarted","Data":"31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f"}
Sep 19 05:58:04.369026 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.368857    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:04.370604 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.370582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl" event={"ID":"81f20ac1-3b85-4c4f-824d-d52972474316","Type":"ContainerStarted","Data":"5db718514360bd49246ff732b800e96024d7f7db8b7e59b0292f7ffea9af31b0"}
Sep 19 05:58:04.370713 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.370618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl" event={"ID":"81f20ac1-3b85-4c4f-824d-d52972474316","Type":"ContainerStarted","Data":"3de61f892ed6524860902bc0a2f188f89888cda5e4889fb5b05748c5f06d6756"}
Sep 19 05:58:04.408469 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.408415    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl" podStartSLOduration=1.408401888 podStartE2EDuration="1.408401888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:58:04.406251897 +0000 UTC m=+1030.016931904" watchObservedRunningTime="2026-09-19 05:58:04.408401888 +0000 UTC m=+1030.019081852"
Sep 19 05:58:04.409048 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.409003    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc" podStartSLOduration=2.4089926249999998 podStartE2EDuration="2.408992625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:58:04.385915464 +0000 UTC m=+1029.996595465" watchObservedRunningTime="2026-09-19 05:58:04.408992625 +0000 UTC m=+1030.019672590"
Sep 19 05:58:04.736974 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:04.736878    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:05.741929 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:05.741890    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:06.381619 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:06.381587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:06.382416 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:06.382395    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:58:07.910182 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.910118    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:58:07.976770 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.976742    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-credential-socket\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.976788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-istio-token\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.976840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr2xk\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-kube-api-access-rr2xk\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.976869    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-certs\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.976906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-envoy\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.976976    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753654e5-1c27-48bc-897d-de9180dea5de-istiod-ca-cert\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977017    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:58:07.977075 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-socket\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977086    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-data\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977111    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753654e5-1c27-48bc-897d-de9180dea5de-istio-podinfo\") pod \"753654e5-1c27-48bc-897d-de9180dea5de\" (UID: \"753654e5-1c27-48bc-897d-de9180dea5de\") "
Sep 19 05:58:07.977501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977273    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:58:07.977501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977327    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:58:07.977501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977485    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/753654e5-1c27-48bc-897d-de9180dea5de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:58:07.977501 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977478    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-data" (OuterVolumeSpecName: "istio-data") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:58:07.978000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977587    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:07.978000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977602    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-workload-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:07.978000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.977611    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-credential-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:07.979314 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.979289    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-kube-api-access-rr2xk" (OuterVolumeSpecName: "kube-api-access-rr2xk") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "kube-api-access-rr2xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:58:07.979314 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.979305    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:58:07.979478 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.979384    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-istio-token" (OuterVolumeSpecName: "istio-token") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:58:07.979610 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:07.979587    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/753654e5-1c27-48bc-897d-de9180dea5de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "753654e5-1c27-48bc-897d-de9180dea5de" (UID: "753654e5-1c27-48bc-897d-de9180dea5de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 05:58:08.078625 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.078595    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:08.078625 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.078622    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr2xk\" (UniqueName: \"kubernetes.io/projected/753654e5-1c27-48bc-897d-de9180dea5de-kube-api-access-rr2xk\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:08.078790 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.078632    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-envoy\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:08.078790 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.078640    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753654e5-1c27-48bc-897d-de9180dea5de-istiod-ca-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:08.078790 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.078647    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753654e5-1c27-48bc-897d-de9180dea5de-istio-data\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:08.078790 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.078655    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753654e5-1c27-48bc-897d-de9180dea5de-istio-podinfo\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:08.398231 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.398194    2581 generic.go:356] "Generic (PLEG): container finished" podID="753654e5-1c27-48bc-897d-de9180dea5de" containerID="aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89" exitCode=0
Sep 19 05:58:08.398428 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.398251    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"
Sep 19 05:58:08.398428 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.398294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" event={"ID":"753654e5-1c27-48bc-897d-de9180dea5de","Type":"ContainerDied","Data":"aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89"}
Sep 19 05:58:08.398428 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.398363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp" event={"ID":"753654e5-1c27-48bc-897d-de9180dea5de","Type":"ContainerDied","Data":"4e40bf3c410a4e5d5c962a34201339fea97cb6d8e0c50e1145f74c84c898aaf4"}
Sep 19 05:58:08.398428 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.398391    2581 scope.go:117] "RemoveContainer" containerID="aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89"
Sep 19 05:58:08.408328 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.408311    2581 scope.go:117] "RemoveContainer" containerID="492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"
Sep 19 05:58:08.417054 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.417023    2581 scope.go:117] "RemoveContainer" containerID="aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89"
Sep 19 05:58:08.417307 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:58:08.417288    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89\": container with ID starting with aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89 not found: ID does not exist" containerID="aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89"
Sep 19 05:58:08.417384 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.417317    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89"} err="failed to get container status \"aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89\": rpc error: code = NotFound desc = could not find container \"aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89\": container with ID starting with aaa9b7b08548929f4b2c1d7e5d78bd83bdd52fc894ace2205f4397f5db7d2c89 not found: ID does not exist"
Sep 19 05:58:08.417384 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.417339    2581 scope.go:117] "RemoveContainer" containerID="492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"
Sep 19 05:58:08.417616 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:58:08.417599    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854\": container with ID starting with 492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854 not found: ID does not exist" containerID="492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"
Sep 19 05:58:08.417677 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.417624    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854"} err="failed to get container status \"492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854\": rpc error: code = NotFound desc = could not find container \"492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854\": container with ID starting with 492bfccd2b19ed271225fd0e0e5ba18fc49004a4f63e9900062cc081965db854 not found: ID does not exist"
Sep 19 05:58:08.421517 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.421496    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"]
Sep 19 05:58:08.425298 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.425276    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-875220-openshift-default-8455794656-lnfxp"]
Sep 19 05:58:08.477638 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:08.477613    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="753654e5-1c27-48bc-897d-de9180dea5de" path="/var/lib/kubelet/pods/753654e5-1c27-48bc-897d-de9180dea5de/volumes"
Sep 19 05:58:14.731754 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.731717    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"]
Sep 19 05:58:14.732212 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.732180    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy"
Sep 19 05:58:14.732212 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.732198    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy"
Sep 19 05:58:14.732388 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.732271    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy"
Sep 19 05:58:14.732388 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.732283    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy"
Sep 19 05:58:14.732497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.732402    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy"
Sep 19 05:58:14.732497 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.732413    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="753654e5-1c27-48bc-897d-de9180dea5de" containerName="istio-proxy"
Sep 19 05:58:14.735482 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.735464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.738021 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.737997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"e2e-conflict-a-95222d07-kserve-self-signed-certs\""
Sep 19 05:58:14.744707 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.744684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"]
Sep 19 05:58:14.843563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.843493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-tmp-dir\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.843563 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.843566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-model-cache\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.843832 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.843591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-dshm\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.843832 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.843632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qtkg\" (UniqueName: \"kubernetes.io/projected/ae388a09-e072-4fd7-ade6-3011f5fd113b-kube-api-access-5qtkg\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.843832 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.843726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-home\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.843832 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.843787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae388a09-e072-4fd7-ade6-3011f5fd113b-tls-certs\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.944589 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-home\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.944753 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae388a09-e072-4fd7-ade6-3011f5fd113b-tls-certs\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.944753 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-tmp-dir\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.944753 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-model-cache\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.944753 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-dshm\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.945009 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qtkg\" (UniqueName: \"kubernetes.io/projected/ae388a09-e072-4fd7-ade6-3011f5fd113b-kube-api-access-5qtkg\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.945009 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.944961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-home\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.945151 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.945110    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-tmp-dir\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.945151 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.945138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-model-cache\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.947192 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.947169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-dshm\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.947346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.947328    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae388a09-e072-4fd7-ade6-3011f5fd113b-tls-certs\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:14.952452 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:14.952433    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qtkg\" (UniqueName: \"kubernetes.io/projected/ae388a09-e072-4fd7-ade6-3011f5fd113b-kube-api-access-5qtkg\") pod \"e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:15.045514 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:15.045479    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:15.315948 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:15.315922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"]
Sep 19 05:58:15.317707 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:58:15.317677    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae388a09_e072_4fd7_ade6_3011f5fd113b.slice/crio-7cc06910b71167a904d7c2af13957486e25c80fdced41ca5ebcbf04483f81197 WatchSource:0}: Error finding container 7cc06910b71167a904d7c2af13957486e25c80fdced41ca5ebcbf04483f81197: Status 404 returned error can't find the container with id 7cc06910b71167a904d7c2af13957486e25c80fdced41ca5ebcbf04483f81197
Sep 19 05:58:15.378469 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:15.378449    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:15.437295 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:15.437259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv" event={"ID":"ae388a09-e072-4fd7-ade6-3011f5fd113b","Type":"ContainerStarted","Data":"ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa"}
Sep 19 05:58:15.437458 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:15.437318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv" event={"ID":"ae388a09-e072-4fd7-ade6-3011f5fd113b","Type":"ContainerStarted","Data":"7cc06910b71167a904d7c2af13957486e25c80fdced41ca5ebcbf04483f81197"}
Sep 19 05:58:15.455683 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:15.455630    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv" podStartSLOduration=1.455614203 podStartE2EDuration="1.455614203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:58:15.453453071 +0000 UTC m=+1041.064133077" watchObservedRunningTime="2026-09-19 05:58:15.455614203 +0000 UTC m=+1041.066294166"
Sep 19 05:58:21.232009 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:21.231972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:58:25.046244 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:25.046208    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:25.046730 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:25.046262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:25.059016 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:25.058987    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:25.505266 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:25.505188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 05:58:38.453401 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.453307    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"]
Sep 19 05:58:38.458235 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.458200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.460994 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.460969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-2d8e5f\"/\"kube-root-ca.crt\""
Sep 19 05:58:38.462307 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.462264    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-2d8e5f\"/\"default-dockercfg-pw8tr\""
Sep 19 05:58:38.462426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.462309    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-2d8e5f\"/\"openshift-service-ca.crt\""
Sep 19 05:58:38.462426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.462288    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-2d8e5f\"/\"noten-model-2d8e5f-kserve-self-signed-certs\""
Sep 19 05:58:38.467980 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.467960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"]
Sep 19 05:58:38.570140 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.570108    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-home\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.570331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.570191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-model-cache\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.570331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.570215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-tmp-dir\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.570331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.570300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf90db-c337-4a09-8cba-37cb269ac270-tls-certs\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.570331 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.570325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdxrk\" (UniqueName: \"kubernetes.io/projected/70cf90db-c337-4a09-8cba-37cb269ac270-kube-api-access-bdxrk\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.570536 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.570351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-dshm\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.670991 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.670952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf90db-c337-4a09-8cba-37cb269ac270-tls-certs\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671193 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdxrk\" (UniqueName: \"kubernetes.io/projected/70cf90db-c337-4a09-8cba-37cb269ac270-kube-api-access-bdxrk\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671193 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-dshm\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671193 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-home\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671372 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-model-cache\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671372 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-tmp-dir\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671561 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-home\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671634 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-model-cache\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.671735 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.671714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-tmp-dir\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.673734 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.673709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-dshm\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.673851 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.673710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf90db-c337-4a09-8cba-37cb269ac270-tls-certs\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.679426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.679404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdxrk\" (UniqueName: \"kubernetes.io/projected/70cf90db-c337-4a09-8cba-37cb269ac270-kube-api-access-bdxrk\") pod \"noten-model-2d8e5f-kserve-576c57d78c-xmxlk\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") " pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:38.769493 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:38.769403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:39.042806 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:39.042765    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"]
Sep 19 05:58:39.044271 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:58:39.044242    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70cf90db_c337_4a09_8cba_37cb269ac270.slice/crio-0b56341479e4c8a9cbbcfe69cd7efa4b443cbf98bad4ca2c71e7b17d6fb421a2 WatchSource:0}: Error finding container 0b56341479e4c8a9cbbcfe69cd7efa4b443cbf98bad4ca2c71e7b17d6fb421a2: Status 404 returned error can't find the container with id 0b56341479e4c8a9cbbcfe69cd7efa4b443cbf98bad4ca2c71e7b17d6fb421a2
Sep 19 05:58:39.569562 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:39.569523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk" event={"ID":"70cf90db-c337-4a09-8cba-37cb269ac270","Type":"ContainerStarted","Data":"94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67"}
Sep 19 05:58:39.569913 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:39.569572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk" event={"ID":"70cf90db-c337-4a09-8cba-37cb269ac270","Type":"ContainerStarted","Data":"0b56341479e4c8a9cbbcfe69cd7efa4b443cbf98bad4ca2c71e7b17d6fb421a2"}
Sep 19 05:58:39.591594 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:39.591537    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk" podStartSLOduration=1.591517385 podStartE2EDuration="1.591517385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:58:39.588285585 +0000 UTC m=+1065.198965594" watchObservedRunningTime="2026-09-19 05:58:39.591517385 +0000 UTC m=+1065.202197350"
Sep 19 05:58:46.226268 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.226231    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"]
Sep 19 05:58:46.226752 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.226604    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc" podUID="72b917e5-a12f-41bd-b68c-7f7435bc7ff0" containerName="payload-pre-processing" containerID="cri-o://31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f" gracePeriod=30
Sep 19 05:58:46.493325 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.493300    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:46.548602 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.548575    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-plugins-config-volume\") pod \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") "
Sep 19 05:58:46.548748 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.548645    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fpvz\" (UniqueName: \"kubernetes.io/projected/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-kube-api-access-9fpvz\") pod \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\" (UID: \"72b917e5-a12f-41bd-b68c-7f7435bc7ff0\") "
Sep 19 05:58:46.548974 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.548953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72b917e5-a12f-41bd-b68c-7f7435bc7ff0" (UID: "72b917e5-a12f-41bd-b68c-7f7435bc7ff0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:58:46.550952 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.550928    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-kube-api-access-9fpvz" (OuterVolumeSpecName: "kube-api-access-9fpvz") pod "72b917e5-a12f-41bd-b68c-7f7435bc7ff0" (UID: "72b917e5-a12f-41bd-b68c-7f7435bc7ff0"). InnerVolumeSpecName "kube-api-access-9fpvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:58:46.607788 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.607756    2581 generic.go:356] "Generic (PLEG): container finished" podID="72b917e5-a12f-41bd-b68c-7f7435bc7ff0" containerID="31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f" exitCode=0
Sep 19 05:58:46.607926 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.607795    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"
Sep 19 05:58:46.607926 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.607796    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc" event={"ID":"72b917e5-a12f-41bd-b68c-7f7435bc7ff0","Type":"ContainerDied","Data":"31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f"}
Sep 19 05:58:46.608018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.607922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc" event={"ID":"72b917e5-a12f-41bd-b68c-7f7435bc7ff0","Type":"ContainerDied","Data":"b1c3d9e4de53b35ef36eab6d63b1f4f8524b4fc1946b82ebe5333994093e94e7"}
Sep 19 05:58:46.608018 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.607957    2581 scope.go:117] "RemoveContainer" containerID="31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f"
Sep 19 05:58:46.617472 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.617456    2581 scope.go:117] "RemoveContainer" containerID="31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f"
Sep 19 05:58:46.617714 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:58:46.617694    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f\": container with ID starting with 31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f not found: ID does not exist" containerID="31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f"
Sep 19 05:58:46.617776 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.617726    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f"} err="failed to get container status \"31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f\": rpc error: code = NotFound desc = could not find container \"31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f\": container with ID starting with 31cc23d9dd8b9afad86564f8b20bd8ed33f610c99c1cf1d51e50caf82b6f836f not found: ID does not exist"
Sep 19 05:58:46.631527 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.631504    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"]
Sep 19 05:58:46.633722 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.633703    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aec2317b-6bb6767ddb-tx4xc"]
Sep 19 05:58:46.649656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.649636    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fpvz\" (UniqueName: \"kubernetes.io/projected/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-kube-api-access-9fpvz\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:46.649656 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:46.649657    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b917e5-a12f-41bd-b68c-7f7435bc7ff0-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:58:48.477308 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:48.477276    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72b917e5-a12f-41bd-b68c-7f7435bc7ff0" path="/var/lib/kubelet/pods/72b917e5-a12f-41bd-b68c-7f7435bc7ff0/volumes"
Sep 19 05:58:48.769558 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:48.769475    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:48.769558 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:48.769536    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:48.782618 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:48.782589    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:58:49.150591 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:49.150557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:58:49.638129 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:58:49.638097    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:59:12.610483 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:12.610445    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"]
Sep 19 05:59:12.610995 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:12.610695    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl" podUID="81f20ac1-3b85-4c4f-824d-d52972474316" containerName="istio-proxy" containerID="cri-o://5db718514360bd49246ff732b800e96024d7f7db8b7e59b0292f7ffea9af31b0" gracePeriod=30
Sep 19 05:59:14.491935 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.491892    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"]
Sep 19 05:59:14.492457 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.492436    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72b917e5-a12f-41bd-b68c-7f7435bc7ff0" containerName="payload-pre-processing"
Sep 19 05:59:14.492526 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.492460    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b917e5-a12f-41bd-b68c-7f7435bc7ff0" containerName="payload-pre-processing"
Sep 19 05:59:14.492581 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.492571    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="72b917e5-a12f-41bd-b68c-7f7435bc7ff0" containerName="payload-pre-processing"
Sep 19 05:59:14.495860 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.495830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.498316 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.498294    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2b3fb3-dockercfg-tjnjw\""
Sep 19 05:59:14.498454 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.498435    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2b3fb3\""
Sep 19 05:59:14.514400 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.514377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"]
Sep 19 05:59:14.622076 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.622023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hcfz\" (UniqueName: \"kubernetes.io/projected/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-kube-api-access-5hcfz\") pod \"payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.622239 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.622102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.723637 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.723599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hcfz\" (UniqueName: \"kubernetes.io/projected/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-kube-api-access-5hcfz\") pod \"payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.723822 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.723662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.724413 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.724393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.738792 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.738765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hcfz\" (UniqueName: \"kubernetes.io/projected/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-kube-api-access-5hcfz\") pod \"payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:14.806188 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:14.806160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:15.076847 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:15.076823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"]
Sep 19 05:59:15.079561 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:59:15.079532    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69baeff6_8513_4b8f_9a61_9fa599e5f8b6.slice/crio-fca3c03d125e6d153e0fbe53f31eaeba044dd1d7577420149c2c262392f570ff WatchSource:0}: Error finding container fca3c03d125e6d153e0fbe53f31eaeba044dd1d7577420149c2c262392f570ff: Status 404 returned error can't find the container with id fca3c03d125e6d153e0fbe53f31eaeba044dd1d7577420149c2c262392f570ff
Sep 19 05:59:15.452484 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:15.452393    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:59:15.765074 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:15.764967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq" event={"ID":"69baeff6-8513-4b8f-9a61-9fa599e5f8b6","Type":"ContainerStarted","Data":"b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4"}
Sep 19 05:59:15.765074 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:15.765022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq" event={"ID":"69baeff6-8513-4b8f-9a61-9fa599e5f8b6","Type":"ContainerStarted","Data":"fca3c03d125e6d153e0fbe53f31eaeba044dd1d7577420149c2c262392f570ff"}
Sep 19 05:59:15.765074 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:15.765069    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:15.783268 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:15.783225    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq" podStartSLOduration=1.783211149 podStartE2EDuration="1.783211149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:15.78073147 +0000 UTC m=+1101.391411452" watchObservedRunningTime="2026-09-19 05:59:15.783211149 +0000 UTC m=+1101.393891176"
Sep 19 05:59:17.778422 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:17.778386    2581 generic.go:356] "Generic (PLEG): container finished" podID="81f20ac1-3b85-4c4f-824d-d52972474316" containerID="5db718514360bd49246ff732b800e96024d7f7db8b7e59b0292f7ffea9af31b0" exitCode=0
Sep 19 05:59:17.778739 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:17.778459    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl" event={"ID":"81f20ac1-3b85-4c4f-824d-d52972474316","Type":"ContainerDied","Data":"5db718514360bd49246ff732b800e96024d7f7db8b7e59b0292f7ffea9af31b0"}
Sep 19 05:59:17.868234 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:17.868213    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:59:17.956956 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:17.956884    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"]
Sep 19 05:59:18.058083 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058051    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-socket\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-certs\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058123    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9szh7\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-kube-api-access-9szh7\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058167    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-credential-socket\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058189    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-istio-token\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058210    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f20ac1-3b85-4c4f-824d-d52972474316-istiod-ca-cert\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058274 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058250    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-data\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058344    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:18.058564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:18.058564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f20ac1-3b85-4c4f-824d-d52972474316-istio-podinfo\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058462    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-envoy\") pod \"81f20ac1-3b85-4c4f-824d-d52972474316\" (UID: \"81f20ac1-3b85-4c4f-824d-d52972474316\") "
Sep 19 05:59:18.058564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058537    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:18.058564 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058555    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-data" (OuterVolumeSpecName: "istio-data") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:18.058872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058678    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81f20ac1-3b85-4c4f-824d-d52972474316-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:59:18.059134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.058968    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.059134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.059060    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-workload-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.059134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.059076    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-credential-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.059134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.059090    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81f20ac1-3b85-4c4f-824d-d52972474316-istiod-ca-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.059134 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.059102    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-data\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.060689 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.060658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-istio-token" (OuterVolumeSpecName: "istio-token") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:59:18.060689 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.060668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81f20ac1-3b85-4c4f-824d-d52972474316-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 05:59:18.060813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.060741    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-kube-api-access-9szh7" (OuterVolumeSpecName: "kube-api-access-9szh7") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "kube-api-access-9szh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:59:18.060813 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.060754    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "81f20ac1-3b85-4c4f-824d-d52972474316" (UID: "81f20ac1-3b85-4c4f-824d-d52972474316"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:18.159771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.159735    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9szh7\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-kube-api-access-9szh7\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.159771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.159767    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81f20ac1-3b85-4c4f-824d-d52972474316-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.159771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.159778    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81f20ac1-3b85-4c4f-824d-d52972474316-istio-podinfo\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.160015 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.159785    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81f20ac1-3b85-4c4f-824d-d52972474316-istio-envoy\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:18.785120 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.785092    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"
Sep 19 05:59:18.785522 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.785131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl" event={"ID":"81f20ac1-3b85-4c4f-824d-d52972474316","Type":"ContainerDied","Data":"3de61f892ed6524860902bc0a2f188f89888cda5e4889fb5b05748c5f06d6756"}
Sep 19 05:59:18.785522 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.785191    2581 scope.go:117] "RemoveContainer" containerID="5db718514360bd49246ff732b800e96024d7f7db8b7e59b0292f7ffea9af31b0"
Sep 19 05:59:18.804754 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.804726    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"]
Sep 19 05:59:18.809432 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:18.809409    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-2d8e5f-openshift-default-c858df59b-ntwvl"]
Sep 19 05:59:20.477480 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:20.477448    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f20ac1-3b85-4c4f-824d-d52972474316" path="/var/lib/kubelet/pods/81f20ac1-3b85-4c4f-824d-d52972474316/volumes"
Sep 19 05:59:26.771553 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:26.771519    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 05:59:32.976939 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:32.976881    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk" podUID="70cf90db-c337-4a09-8cba-37cb269ac270" containerName="main" containerID="cri-o://94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67" gracePeriod=45
Sep 19 05:59:33.241115 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.241059    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:59:33.294094 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.293912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-tmp-dir\") pod \"70cf90db-c337-4a09-8cba-37cb269ac270\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") "
Sep 19 05:59:33.294094 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.293973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf90db-c337-4a09-8cba-37cb269ac270-tls-certs\") pod \"70cf90db-c337-4a09-8cba-37cb269ac270\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") "
Sep 19 05:59:33.294094 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.294061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdxrk\" (UniqueName: \"kubernetes.io/projected/70cf90db-c337-4a09-8cba-37cb269ac270-kube-api-access-bdxrk\") pod \"70cf90db-c337-4a09-8cba-37cb269ac270\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") "
Sep 19 05:59:33.294414 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.294109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-home\") pod \"70cf90db-c337-4a09-8cba-37cb269ac270\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") "
Sep 19 05:59:33.294414 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.294154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-dshm\") pod \"70cf90db-c337-4a09-8cba-37cb269ac270\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") "
Sep 19 05:59:33.294414 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.294205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-model-cache\") pod \"70cf90db-c337-4a09-8cba-37cb269ac270\" (UID: \"70cf90db-c337-4a09-8cba-37cb269ac270\") "
Sep 19 05:59:33.295571 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.295160    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70cf90db-c337-4a09-8cba-37cb269ac270" (UID: "70cf90db-c337-4a09-8cba-37cb269ac270"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:33.296778 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.296610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-home" (OuterVolumeSpecName: "home") pod "70cf90db-c337-4a09-8cba-37cb269ac270" (UID: "70cf90db-c337-4a09-8cba-37cb269ac270"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:33.297224 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.296950    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-model-cache" (OuterVolumeSpecName: "model-cache") pod "70cf90db-c337-4a09-8cba-37cb269ac270" (UID: "70cf90db-c337-4a09-8cba-37cb269ac270"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:33.309785 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.309755    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70cf90db-c337-4a09-8cba-37cb269ac270-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70cf90db-c337-4a09-8cba-37cb269ac270" (UID: "70cf90db-c337-4a09-8cba-37cb269ac270"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:59:33.311468 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.311444    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70cf90db-c337-4a09-8cba-37cb269ac270-kube-api-access-bdxrk" (OuterVolumeSpecName: "kube-api-access-bdxrk") pod "70cf90db-c337-4a09-8cba-37cb269ac270" (UID: "70cf90db-c337-4a09-8cba-37cb269ac270"). InnerVolumeSpecName "kube-api-access-bdxrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:59:33.311667 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.311652    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-dshm" (OuterVolumeSpecName: "dshm") pod "70cf90db-c337-4a09-8cba-37cb269ac270" (UID: "70cf90db-c337-4a09-8cba-37cb269ac270"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:59:33.396153 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.396123    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:33.396153 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.396149    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:33.396153 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.396158    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:33.396346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.396167    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70cf90db-c337-4a09-8cba-37cb269ac270-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:33.396346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.396175    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70cf90db-c337-4a09-8cba-37cb269ac270-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:33.396346 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.396183    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdxrk\" (UniqueName: \"kubernetes.io/projected/70cf90db-c337-4a09-8cba-37cb269ac270-kube-api-access-bdxrk\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 05:59:33.864516 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.864482    2581 generic.go:356] "Generic (PLEG): container finished" podID="70cf90db-c337-4a09-8cba-37cb269ac270" containerID="94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67" exitCode=0
Sep 19 05:59:33.864516 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.864522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk" event={"ID":"70cf90db-c337-4a09-8cba-37cb269ac270","Type":"ContainerDied","Data":"94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67"}
Sep 19 05:59:33.864752 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.864544    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"
Sep 19 05:59:33.864752 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.864559    2581 scope.go:117] "RemoveContainer" containerID="94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67"
Sep 19 05:59:33.864752 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.864550    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk" event={"ID":"70cf90db-c337-4a09-8cba-37cb269ac270","Type":"ContainerDied","Data":"0b56341479e4c8a9cbbcfe69cd7efa4b443cbf98bad4ca2c71e7b17d6fb421a2"}
Sep 19 05:59:33.874109 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.874085    2581 scope.go:117] "RemoveContainer" containerID="94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67"
Sep 19 05:59:33.874387 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:59:33.874366    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67\": container with ID starting with 94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67 not found: ID does not exist" containerID="94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67"
Sep 19 05:59:33.874429 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.874394    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67"} err="failed to get container status \"94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67\": rpc error: code = NotFound desc = could not find container \"94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67\": container with ID starting with 94ff44811acdc28fbd0edcf53835664d3ded61bad48e7e9c7d37afa3d7849a67 not found: ID does not exist"
Sep 19 05:59:33.887135 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.887113    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"]
Sep 19 05:59:33.893267 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:33.893245    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-2d8e5f/noten-model-2d8e5f-kserve-576c57d78c-xmxlk"]
Sep 19 05:59:34.477100 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:34.477066    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70cf90db-c337-4a09-8cba-37cb269ac270" path="/var/lib/kubelet/pods/70cf90db-c337-4a09-8cba-37cb269ac270/volumes"
Sep 19 05:59:40.661791 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.661753    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 05:59:40.835699 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.835659    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"]
Sep 19 05:59:40.836195 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.836174    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f20ac1-3b85-4c4f-824d-d52972474316" containerName="istio-proxy"
Sep 19 05:59:40.836195 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.836195    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f20ac1-3b85-4c4f-824d-d52972474316" containerName="istio-proxy"
Sep 19 05:59:40.836394 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.836217    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70cf90db-c337-4a09-8cba-37cb269ac270" containerName="main"
Sep 19 05:59:40.836394 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.836225    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70cf90db-c337-4a09-8cba-37cb269ac270" containerName="main"
Sep 19 05:59:40.836394 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.836332    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f20ac1-3b85-4c4f-824d-d52972474316" containerName="istio-proxy"
Sep 19 05:59:40.836394 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.836347    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70cf90db-c337-4a09-8cba-37cb269ac270" containerName="main"
Sep 19 05:59:40.839777 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.839755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.842356 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.842334    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-836c16-openshift-default-dockercfg-tsvnn\""
Sep 19 05:59:40.842480 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.842358    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 05:59:40.854144 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.854123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"]
Sep 19 05:59:40.970426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-data\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-envoy\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970426 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-token\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-socket\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/607484dc-1c8a-4094-9b21-7741ea3c4f53-istiod-ca-cert\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-certs\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-credential-socket\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-podinfo\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:40.970771 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:40.970675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqv5d\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-kube-api-access-tqv5d\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071177 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-certs\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071177 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-credential-socket\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071407 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-podinfo\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071407 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqv5d\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-kube-api-access-tqv5d\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071407 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-data\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071407 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-envoy\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071407 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-token\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071407 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-socket\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071731 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/607484dc-1c8a-4094-9b21-7741ea3c4f53-istiod-ca-cert\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071731 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-certs\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071731 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-data\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071731 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071682    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-credential-socket\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.071969 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.071912    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-socket\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.072058 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.072021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/607484dc-1c8a-4094-9b21-7741ea3c4f53-istiod-ca-cert\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.073728 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.073709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-podinfo\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.073871 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.073849    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-envoy\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.080131 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.080100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqv5d\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-kube-api-access-tqv5d\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.080258 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.080201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-token\") pod \"e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") " pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.150635 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.150603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:41.427612 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.427587    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"]
Sep 19 05:59:41.429325 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:59:41.429297    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607484dc_1c8a_4094_9b21_7741ea3c4f53.slice/crio-ed88f3f74345ef649c238e99621ceaf15413b472b0048538ab0b9844559dfbb4 WatchSource:0}: Error finding container ed88f3f74345ef649c238e99621ceaf15413b472b0048538ab0b9844559dfbb4: Status 404 returned error can't find the container with id ed88f3f74345ef649c238e99621ceaf15413b472b0048538ab0b9844559dfbb4
Sep 19 05:59:41.431840 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.431803    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:59:41.431950 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.431906    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:59:41.432000 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.431955    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 05:59:41.913851 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.913808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" event={"ID":"607484dc-1c8a-4094-9b21-7741ea3c4f53","Type":"ContainerStarted","Data":"f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"}
Sep 19 05:59:41.914322 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.913873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" event={"ID":"607484dc-1c8a-4094-9b21-7741ea3c4f53","Type":"ContainerStarted","Data":"ed88f3f74345ef649c238e99621ceaf15413b472b0048538ab0b9844559dfbb4"}
Sep 19 05:59:41.939049 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:41.938992    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podStartSLOduration=1.9389766370000001 podStartE2EDuration="1.938976637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:41.937452065 +0000 UTC m=+1127.548132227" watchObservedRunningTime="2026-09-19 05:59:41.938976637 +0000 UTC m=+1127.549656600"
Sep 19 05:59:42.151627 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:42.151594    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:42.156806 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:42.156781    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:42.919292 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:42.919252    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:42.920136 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:42.920113    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 05:59:48.776386 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.776343    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"]
Sep 19 05:59:48.780111 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.780089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.783491 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.783456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-836c16-serving-cert\""
Sep 19 05:59:48.790140 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.789962    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"]
Sep 19 05:59:48.849218 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.849180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx4t7\" (UniqueName: \"kubernetes.io/projected/7b919a27-9c80-45f9-b898-d70afcc92af8-kube-api-access-dx4t7\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.849412 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.849225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.849412 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.849266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.849412 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.849374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b919a27-9c80-45f9-b898-d70afcc92af8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.950179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.950145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx4t7\" (UniqueName: \"kubernetes.io/projected/7b919a27-9c80-45f9-b898-d70afcc92af8-kube-api-access-dx4t7\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.950179 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.950184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.950382 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.950219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.950382 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:59:48.950306    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-serving-cert: secret "maas-api-e2e-shared-b-w2-836c16-serving-cert" not found
Sep 19 05:59:48.950382 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.950320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b919a27-9c80-45f9-b898-d70afcc92af8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.950491 ip-10-0-131-202 kubenswrapper[2581]: E0919 05:59:48.950387    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls podName:7b919a27-9c80-45f9-b898-d70afcc92af8 nodeName:}" failed. No retries permitted until 2026-09-19 05:59:49.450366847 +0000 UTC m=+1135.061046803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls") pod "maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" (UID: "7b919a27-9c80-45f9-b898-d70afcc92af8") : secret "maas-api-e2e-shared-b-w2-836c16-serving-cert" not found
Sep 19 05:59:48.950955 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.950938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b919a27-9c80-45f9-b898-d70afcc92af8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.952800 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.952770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:48.958924 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:48.958907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx4t7\" (UniqueName: \"kubernetes.io/projected/7b919a27-9c80-45f9-b898-d70afcc92af8-kube-api-access-dx4t7\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:49.456165 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.456134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:49.458706 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.458676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:49.520363 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.520326    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"]
Sep 19 05:59:49.524149 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.524122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.529187 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.529166    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-836c16-dockercfg-8znf6\""
Sep 19 05:59:49.529301 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.529204    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-836c16\""
Sep 19 05:59:49.538500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.538477    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"]
Sep 19 05:59:49.657966 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.657928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpg9g\" (UniqueName: \"kubernetes.io/projected/c350701a-3d27-4e67-a454-e5a422bf1897-kube-api-access-mpg9g\") pod \"payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.658183 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.657993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c350701a-3d27-4e67-a454-e5a422bf1897-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.696374 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.696337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:49.759303 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.759265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpg9g\" (UniqueName: \"kubernetes.io/projected/c350701a-3d27-4e67-a454-e5a422bf1897-kube-api-access-mpg9g\") pod \"payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.759442 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.759366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c350701a-3d27-4e67-a454-e5a422bf1897-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.760265 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.760234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c350701a-3d27-4e67-a454-e5a422bf1897-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.768500 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.768474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpg9g\" (UniqueName: \"kubernetes.io/projected/c350701a-3d27-4e67-a454-e5a422bf1897-kube-api-access-mpg9g\") pod \"payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.834952 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.834923    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:49.971961 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.971937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"]
Sep 19 05:59:49.973457 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:59:49.973431    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b919a27_9c80_45f9_b898_d70afcc92af8.slice/crio-c517f7bef216fbd23640b2ca301a958a07b153c43c558da71aaf4364922ec170 WatchSource:0}: Error finding container c517f7bef216fbd23640b2ca301a958a07b153c43c558da71aaf4364922ec170: Status 404 returned error can't find the container with id c517f7bef216fbd23640b2ca301a958a07b153c43c558da71aaf4364922ec170
Sep 19 05:59:49.974798 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:49.974778    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:59:50.107143 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.107116    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"]
Sep 19 05:59:50.108891 ip-10-0-131-202 kubenswrapper[2581]: W0919 05:59:50.108866    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc350701a_3d27_4e67_a454_e5a422bf1897.slice/crio-ab6c412b3a7860c5d0e51b8bcc482a7a8abc61ed7d91a74549745e8015a16a2e WatchSource:0}: Error finding container ab6c412b3a7860c5d0e51b8bcc482a7a8abc61ed7d91a74549745e8015a16a2e: Status 404 returned error can't find the container with id ab6c412b3a7860c5d0e51b8bcc482a7a8abc61ed7d91a74549745e8015a16a2e
Sep 19 05:59:50.963028 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.962987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f" event={"ID":"c350701a-3d27-4e67-a454-e5a422bf1897","Type":"ContainerStarted","Data":"64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f"}
Sep 19 05:59:50.963498 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.963087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f" event={"ID":"c350701a-3d27-4e67-a454-e5a422bf1897","Type":"ContainerStarted","Data":"ab6c412b3a7860c5d0e51b8bcc482a7a8abc61ed7d91a74549745e8015a16a2e"}
Sep 19 05:59:50.963498 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.963122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 05:59:50.964834 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.964802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" event={"ID":"7b919a27-9c80-45f9-b898-d70afcc92af8","Type":"ContainerStarted","Data":"9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf"}
Sep 19 05:59:50.964964 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.964852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" event={"ID":"7b919a27-9c80-45f9-b898-d70afcc92af8","Type":"ContainerStarted","Data":"c517f7bef216fbd23640b2ca301a958a07b153c43c558da71aaf4364922ec170"}
Sep 19 05:59:50.964964 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.964887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 05:59:50.980492 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:50.980440    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f" podStartSLOduration=1.98042159 podStartE2EDuration="1.98042159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:50.979891038 +0000 UTC m=+1136.590571023" watchObservedRunningTime="2026-09-19 05:59:50.98042159 +0000 UTC m=+1136.591101552"
Sep 19 05:59:51.000976 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:51.000920    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" podStartSLOduration=2.588352773 podStartE2EDuration="3.000903649s" totalImagesPullingTime="412.550876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:50.998423768 +0000 UTC m=+1136.609103757" watchObservedRunningTime="2026-09-19 05:59:51.000903649 +0000 UTC m=+1136.611583614"
Sep 19 05:59:56.022013 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:56.021972    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"]
Sep 19 05:59:56.975872 ip-10-0-131-202 kubenswrapper[2581]: I0919 05:59:56.975790    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 06:00:00.145398 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.145359    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"]
Sep 19 06:00:00.150344 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.150324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"
Sep 19 06:00:00.153334 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.153310    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\""
Sep 19 06:00:00.159160 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.159133    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"]
Sep 19 06:00:00.162455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.162435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"
Sep 19 06:00:00.168195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.168172    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"]
Sep 19 06:00:00.172573 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.172551    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"
Sep 19 06:00:00.179549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.179521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"]
Sep 19 06:00:00.203779 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.203752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"
Sep 19 06:00:00.448563 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.448530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"]
Sep 19 06:00:00.450407 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:00:00.450382    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bac417e_9ba6_4ee9_9f38_ba741425695a.slice/crio-2bc952a2939a1c8f19384004d38880a3909e09e30cf53004c702d3682af5e1a0 WatchSource:0}: Error finding container 2bc952a2939a1c8f19384004d38880a3909e09e30cf53004c702d3682af5e1a0: Status 404 returned error can't find the container with id 2bc952a2939a1c8f19384004d38880a3909e09e30cf53004c702d3682af5e1a0
Sep 19 06:00:00.498004 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:00.497979    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"]
Sep 19 06:00:00.500896 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:00:00.500870    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a15535_402c_4e26_9ce2_b991dc2c8ced.slice/crio-57cc5b44f8fe39a18d42995ef02ed7cd3696c340948ded58fd325907f90e5472 WatchSource:0}: Error finding container 57cc5b44f8fe39a18d42995ef02ed7cd3696c340948ded58fd325907f90e5472: Status 404 returned error can't find the container with id 57cc5b44f8fe39a18d42995ef02ed7cd3696c340948ded58fd325907f90e5472
Sep 19 06:00:01.023910 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:01.023874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss" event={"ID":"52a15535-402c-4e26-9ce2-b991dc2c8ced","Type":"ContainerStarted","Data":"57cc5b44f8fe39a18d42995ef02ed7cd3696c340948ded58fd325907f90e5472"}
Sep 19 06:00:01.025146 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:01.025121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7" event={"ID":"2bac417e-9ba6-4ee9-9f38-ba741425695a","Type":"ContainerStarted","Data":"2bc952a2939a1c8f19384004d38880a3909e09e30cf53004c702d3682af5e1a0"}
Sep 19 06:00:01.972691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:01.972664    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 06:00:02.306710 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:02.306672    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:00:05.054757 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:05.054714    2581 generic.go:356] "Generic (PLEG): container finished" podID="2bac417e-9ba6-4ee9-9f38-ba741425695a" containerID="b587fa4536802ed64c8b5fd056a7a1576363a7a12b85678a2b77e298e261d0b3" exitCode=0
Sep 19 06:00:05.055471 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:05.054876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7" event={"ID":"2bac417e-9ba6-4ee9-9f38-ba741425695a","Type":"ContainerDied","Data":"b587fa4536802ed64c8b5fd056a7a1576363a7a12b85678a2b77e298e261d0b3"}
Sep 19 06:00:05.058063 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:05.057998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss" event={"ID":"52a15535-402c-4e26-9ce2-b991dc2c8ced","Type":"ContainerStarted","Data":"1f2172e731b779b1b142e702ee676bb84274e93c751daf97097c37482722625b"}
Sep 19 06:00:05.087892 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:05.087828    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss" podStartSLOduration=1.685701711 podStartE2EDuration="5.087813374s" totalImagesPullingTime="3.402111663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:05.085424556 +0000 UTC m=+1150.696104554" watchObservedRunningTime="2026-09-19 06:00:05.087813374 +0000 UTC m=+1150.698493339"
Sep 19 06:00:06.065141 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:06.065106    2581 generic.go:356] "Generic (PLEG): container finished" podID="52a15535-402c-4e26-9ce2-b991dc2c8ced" containerID="1f2172e731b779b1b142e702ee676bb84274e93c751daf97097c37482722625b" exitCode=0
Sep 19 06:00:06.065559 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:06.065199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss" event={"ID":"52a15535-402c-4e26-9ce2-b991dc2c8ced","Type":"ContainerDied","Data":"1f2172e731b779b1b142e702ee676bb84274e93c751daf97097c37482722625b"}
Sep 19 06:00:06.267713 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:06.267690    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"
Sep 19 06:00:07.071657 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:07.071625    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"
Sep 19 06:00:07.072080 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:07.071655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7" event={"ID":"2bac417e-9ba6-4ee9-9f38-ba741425695a","Type":"ContainerDied","Data":"2bc952a2939a1c8f19384004d38880a3909e09e30cf53004c702d3682af5e1a0"}
Sep 19 06:00:07.072080 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:07.071702    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bc952a2939a1c8f19384004d38880a3909e09e30cf53004c702d3682af5e1a0"
Sep 19 06:00:07.247474 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:07.247443    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"
Sep 19 06:00:08.077597 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:08.077567    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"
Sep 19 06:00:08.078081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:08.077605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss" event={"ID":"52a15535-402c-4e26-9ce2-b991dc2c8ced","Type":"ContainerDied","Data":"57cc5b44f8fe39a18d42995ef02ed7cd3696c340948ded58fd325907f90e5472"}
Sep 19 06:00:08.078081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:08.077648    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57cc5b44f8fe39a18d42995ef02ed7cd3696c340948ded58fd325907f90e5472"
Sep 19 06:00:11.042398 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.042353    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5" podUID="aebe971b-27a5-4447-b147-3eaefbff9f29" containerName="main" containerID="cri-o://c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937" gracePeriod=45
Sep 19 06:00:11.304247 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.304181    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 06:00:11.384698 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384665    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-tmp-dir\") pod \"aebe971b-27a5-4447-b147-3eaefbff9f29\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") "
Sep 19 06:00:11.384698 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384704    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjpdd\" (UniqueName: \"kubernetes.io/projected/aebe971b-27a5-4447-b147-3eaefbff9f29-kube-api-access-gjpdd\") pod \"aebe971b-27a5-4447-b147-3eaefbff9f29\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") "
Sep 19 06:00:11.384962 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384746    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-home\") pod \"aebe971b-27a5-4447-b147-3eaefbff9f29\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") "
Sep 19 06:00:11.384962 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-model-cache\") pod \"aebe971b-27a5-4447-b147-3eaefbff9f29\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") "
Sep 19 06:00:11.384962 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebe971b-27a5-4447-b147-3eaefbff9f29-tls-certs\") pod \"aebe971b-27a5-4447-b147-3eaefbff9f29\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") "
Sep 19 06:00:11.384962 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-dshm\") pod \"aebe971b-27a5-4447-b147-3eaefbff9f29\" (UID: \"aebe971b-27a5-4447-b147-3eaefbff9f29\") "
Sep 19 06:00:11.385207 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.384986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aebe971b-27a5-4447-b147-3eaefbff9f29" (UID: "aebe971b-27a5-4447-b147-3eaefbff9f29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:00:11.385207 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.385053    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-home" (OuterVolumeSpecName: "home") pod "aebe971b-27a5-4447-b147-3eaefbff9f29" (UID: "aebe971b-27a5-4447-b147-3eaefbff9f29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:00:11.385327 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.385228    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-model-cache" (OuterVolumeSpecName: "model-cache") pod "aebe971b-27a5-4447-b147-3eaefbff9f29" (UID: "aebe971b-27a5-4447-b147-3eaefbff9f29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:00:11.385387 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.385369    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:00:11.385442 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.385394    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:00:11.385442 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.385409    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:00:11.387929 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.387901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-dshm" (OuterVolumeSpecName: "dshm") pod "aebe971b-27a5-4447-b147-3eaefbff9f29" (UID: "aebe971b-27a5-4447-b147-3eaefbff9f29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:00:11.388095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.387901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aebe971b-27a5-4447-b147-3eaefbff9f29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aebe971b-27a5-4447-b147-3eaefbff9f29" (UID: "aebe971b-27a5-4447-b147-3eaefbff9f29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:00:11.388167 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.388127    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aebe971b-27a5-4447-b147-3eaefbff9f29-kube-api-access-gjpdd" (OuterVolumeSpecName: "kube-api-access-gjpdd") pod "aebe971b-27a5-4447-b147-3eaefbff9f29" (UID: "aebe971b-27a5-4447-b147-3eaefbff9f29"). InnerVolumeSpecName "kube-api-access-gjpdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:00:11.486195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.486160    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebe971b-27a5-4447-b147-3eaefbff9f29-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:00:11.486195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.486190    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebe971b-27a5-4447-b147-3eaefbff9f29-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:00:11.486195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:11.486202    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjpdd\" (UniqueName: \"kubernetes.io/projected/aebe971b-27a5-4447-b147-3eaefbff9f29-kube-api-access-gjpdd\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:00:12.106135 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.106104    2581 generic.go:356] "Generic (PLEG): container finished" podID="aebe971b-27a5-4447-b147-3eaefbff9f29" containerID="c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937" exitCode=0
Sep 19 06:00:12.106641 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.106147    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"
Sep 19 06:00:12.106641 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.106169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5" event={"ID":"aebe971b-27a5-4447-b147-3eaefbff9f29","Type":"ContainerDied","Data":"c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937"}
Sep 19 06:00:12.106641 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.106225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5" event={"ID":"aebe971b-27a5-4447-b147-3eaefbff9f29","Type":"ContainerDied","Data":"c3964471ca357966de00900f0d14faa7e330e53ee402e30b646897ab9f933430"}
Sep 19 06:00:12.106641 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.106251    2581 scope.go:117] "RemoveContainer" containerID="c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937"
Sep 19 06:00:12.116239 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.116222    2581 scope.go:117] "RemoveContainer" containerID="c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937"
Sep 19 06:00:12.116581 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:00:12.116556    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937\": container with ID starting with c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937 not found: ID does not exist" containerID="c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937"
Sep 19 06:00:12.116653 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.116589    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937"} err="failed to get container status \"c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937\": rpc error: code = NotFound desc = could not find container \"c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937\": container with ID starting with c0995e7ea962572d2a434e0c2b7aeaaddf067ca3563eebdbc8e2af6407fb3937 not found: ID does not exist"
Sep 19 06:00:12.130662 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.130638    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"]
Sep 19 06:00:12.135474 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.135452    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4b78e2/test-model-4b78e2-kserve-74fb9cdb49-27mm5"]
Sep 19 06:00:12.479373 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:12.479298    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aebe971b-27a5-4447-b147-3eaefbff9f29" path="/var/lib/kubelet/pods/aebe971b-27a5-4447-b147-3eaefbff9f29/volumes"
Sep 19 06:00:13.610628 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:13.610596    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:58100->10.133.0.64:15021: read: connection reset by peer" start-of-body=
Sep 19 06:00:13.610999 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:13.610664    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:58100->10.133.0.64:15021: read: connection reset by peer"
Sep 19 06:00:14.119725 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:14.119689    2581 generic.go:356] "Generic (PLEG): container finished" podID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerID="f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498" exitCode=0
Sep 19 06:00:14.119930 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:14.119731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" event={"ID":"607484dc-1c8a-4094-9b21-7741ea3c4f53","Type":"ContainerDied","Data":"f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"}
Sep 19 06:00:14.120195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:14.120175    2581 scope.go:117] "RemoveContainer" containerID="f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"
Sep 19 06:00:14.121354 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:14.121325    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:00:14.121456 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:14.121436    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:00:14.121519 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:14.121488    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:00:15.127493 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:15.127456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" event={"ID":"607484dc-1c8a-4094-9b21-7741ea3c4f53","Type":"ContainerStarted","Data":"daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801"}
Sep 19 06:00:15.151693 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:15.151656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 06:00:15.151884 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:15.151838    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 19 06:00:15.151937 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:15.151909    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 19 06:00:16.151368 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:16.151335    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 19 06:00:16.151857 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:16.151405    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 19 06:00:17.151950 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:17.151910    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 19 06:00:17.152377 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:17.151963    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 19 06:00:18.151951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:18.151912    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 19 06:00:18.152454 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:18.151986    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 19 06:00:19.151254 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:19.151215    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 19 06:00:19.151480 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:19.151270    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 19 06:00:20.150941 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:20.150906    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 19 06:00:20.151428 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:20.150968    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 19 06:00:21.155141 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:21.155107    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 06:00:21.155560 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:21.155536    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 06:00:21.156437 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:21.156418    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 06:00:33.403861 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.403820    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"]
Sep 19 06:00:33.404274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404230    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a15535-402c-4e26-9ce2-b991dc2c8ced" containerName="cleanup"
Sep 19 06:00:33.404274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404243    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a15535-402c-4e26-9ce2-b991dc2c8ced" containerName="cleanup"
Sep 19 06:00:33.404274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404260    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aebe971b-27a5-4447-b147-3eaefbff9f29" containerName="main"
Sep 19 06:00:33.404274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404265    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aebe971b-27a5-4447-b147-3eaefbff9f29" containerName="main"
Sep 19 06:00:33.404274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404273    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bac417e-9ba6-4ee9-9f38-ba741425695a" containerName="cleanup"
Sep 19 06:00:33.404435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404279    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bac417e-9ba6-4ee9-9f38-ba741425695a" containerName="cleanup"
Sep 19 06:00:33.404435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404340    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a15535-402c-4e26-9ce2-b991dc2c8ced" containerName="cleanup"
Sep 19 06:00:33.404435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404348    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aebe971b-27a5-4447-b147-3eaefbff9f29" containerName="main"
Sep 19 06:00:33.404435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.404354    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bac417e-9ba6-4ee9-9f38-ba741425695a" containerName="cleanup"
Sep 19 06:00:33.411748 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.411716    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.414501 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.414480    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-957d1190-dockercfg-xcfhv\""
Sep 19 06:00:33.415970 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.415859    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-957d1190\""
Sep 19 06:00:33.417211 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.417188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"]
Sep 19 06:00:33.604856 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.604822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8f9m\" (UniqueName: \"kubernetes.io/projected/3103d793-b0be-4076-a029-6a476dd1ba20-kube-api-access-m8f9m\") pod \"payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.605063 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.604879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3103d793-b0be-4076-a029-6a476dd1ba20-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.705958 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.705877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8f9m\" (UniqueName: \"kubernetes.io/projected/3103d793-b0be-4076-a029-6a476dd1ba20-kube-api-access-m8f9m\") pod \"payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.705958 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.705951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3103d793-b0be-4076-a029-6a476dd1ba20-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.706663 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.706642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3103d793-b0be-4076-a029-6a476dd1ba20-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.714098 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.714078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8f9m\" (UniqueName: \"kubernetes.io/projected/3103d793-b0be-4076-a029-6a476dd1ba20-kube-api-access-m8f9m\") pod \"payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:33.723947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:33.723926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:34.001876 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:34.001851    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"]
Sep 19 06:00:34.003701 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:00:34.003674    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3103d793_b0be_4076_a029_6a476dd1ba20.slice/crio-9e09983cae5d5e49e1f68e3f0d87d1f8f7166f8cd007fd71e10e3e17a25cf338 WatchSource:0}: Error finding container 9e09983cae5d5e49e1f68e3f0d87d1f8f7166f8cd007fd71e10e3e17a25cf338: Status 404 returned error can't find the container with id 9e09983cae5d5e49e1f68e3f0d87d1f8f7166f8cd007fd71e10e3e17a25cf338
Sep 19 06:00:34.231463 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:34.231415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b" event={"ID":"3103d793-b0be-4076-a029-6a476dd1ba20","Type":"ContainerStarted","Data":"9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3"}
Sep 19 06:00:34.231630 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:34.231484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b" event={"ID":"3103d793-b0be-4076-a029-6a476dd1ba20","Type":"ContainerStarted","Data":"9e09983cae5d5e49e1f68e3f0d87d1f8f7166f8cd007fd71e10e3e17a25cf338"}
Sep 19 06:00:34.231630 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:34.231513    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:34.248899 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:34.248832    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b" podStartSLOduration=1.248817268 podStartE2EDuration="1.248817268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:34.247755862 +0000 UTC m=+1179.858435859" watchObservedRunningTime="2026-09-19 06:00:34.248817268 +0000 UTC m=+1179.859497231"
Sep 19 06:00:44.718461 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:44.718429    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:00:45.239921 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:45.239893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:00:55.128895 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:55.128862    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"]
Sep 19 06:00:56.472479 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:56.472450    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:00:56.475565 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:56.475543    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:00:56.478147 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:56.478124    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:00:56.480356 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:00:56.480337    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:01:07.069028 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:07.068990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:01:08.938748 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:08.938710    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"]
Sep 19 06:01:08.939291 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:08.938942    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w" podUID="554e22bd-2655-4605-b41e-192d357145a4" containerName="payload-pre-processing" containerID="cri-o://bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d" gracePeriod=30
Sep 19 06:01:09.195523 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.195459    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 06:01:09.344147 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.344113    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/554e22bd-2655-4605-b41e-192d357145a4-plugins-config-volume\") pod \"554e22bd-2655-4605-b41e-192d357145a4\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") "
Sep 19 06:01:09.344147 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.344158    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w27rs\" (UniqueName: \"kubernetes.io/projected/554e22bd-2655-4605-b41e-192d357145a4-kube-api-access-w27rs\") pod \"554e22bd-2655-4605-b41e-192d357145a4\" (UID: \"554e22bd-2655-4605-b41e-192d357145a4\") "
Sep 19 06:01:09.344597 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.344570    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554e22bd-2655-4605-b41e-192d357145a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "554e22bd-2655-4605-b41e-192d357145a4" (UID: "554e22bd-2655-4605-b41e-192d357145a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:01:09.346552 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.346528    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554e22bd-2655-4605-b41e-192d357145a4-kube-api-access-w27rs" (OuterVolumeSpecName: "kube-api-access-w27rs") pod "554e22bd-2655-4605-b41e-192d357145a4" (UID: "554e22bd-2655-4605-b41e-192d357145a4"). InnerVolumeSpecName "kube-api-access-w27rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:09.419906 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.419867    2581 generic.go:356] "Generic (PLEG): container finished" podID="554e22bd-2655-4605-b41e-192d357145a4" containerID="bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d" exitCode=0
Sep 19 06:01:09.420091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.419913    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"
Sep 19 06:01:09.420091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.419934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w" event={"ID":"554e22bd-2655-4605-b41e-192d357145a4","Type":"ContainerDied","Data":"bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d"}
Sep 19 06:01:09.420091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.419977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w" event={"ID":"554e22bd-2655-4605-b41e-192d357145a4","Type":"ContainerDied","Data":"f489d4ba25f4dd730ff0cb1b0d5cfe85e771cc65b22d2d13ec1d3906310b74d6"}
Sep 19 06:01:09.420091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.420000    2581 scope.go:117] "RemoveContainer" containerID="bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d"
Sep 19 06:01:09.429481 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.429464    2581 scope.go:117] "RemoveContainer" containerID="bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d"
Sep 19 06:01:09.429747 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:01:09.429726    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d\": container with ID starting with bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d not found: ID does not exist" containerID="bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d"
Sep 19 06:01:09.429815 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.429754    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d"} err="failed to get container status \"bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d\": rpc error: code = NotFound desc = could not find container \"bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d\": container with ID starting with bfb18432371c362b1a00fa8d1417ec03c32da6a5748832356392f4b57174b97d not found: ID does not exist"
Sep 19 06:01:09.441371 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.441347    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"]
Sep 19 06:01:09.445518 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.445500    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c85316-8466b65b58-pzr7w"]
Sep 19 06:01:09.445832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.445790    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/554e22bd-2655-4605-b41e-192d357145a4-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:09.445832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:09.445816    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w27rs\" (UniqueName: \"kubernetes.io/projected/554e22bd-2655-4605-b41e-192d357145a4-kube-api-access-w27rs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.149585 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.149544    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm" podUID="8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" containerName="main" containerID="cri-o://7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a" gracePeriod=45
Sep 19 06:01:10.406915 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.406841    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 06:01:10.427830 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.427790    2581 generic.go:356] "Generic (PLEG): container finished" podID="8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" containerID="7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a" exitCode=0
Sep 19 06:01:10.428012 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.427848    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"
Sep 19 06:01:10.428012 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.427880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm" event={"ID":"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6","Type":"ContainerDied","Data":"7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a"}
Sep 19 06:01:10.428012 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.427918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm" event={"ID":"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6","Type":"ContainerDied","Data":"a62d15d2a3af24a3274dae559f6e43d1645d1b63dab73596aed07ce6dd105a42"}
Sep 19 06:01:10.428012 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.427935    2581 scope.go:117] "RemoveContainer" containerID="7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a"
Sep 19 06:01:10.438946 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.438921    2581 scope.go:117] "RemoveContainer" containerID="7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a"
Sep 19 06:01:10.439347 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:01:10.439323    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a\": container with ID starting with 7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a not found: ID does not exist" containerID="7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a"
Sep 19 06:01:10.439426 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.439360    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a"} err="failed to get container status \"7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a\": rpc error: code = NotFound desc = could not find container \"7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a\": container with ID starting with 7e13210124f02b63ddceb9ec2d98fda12e383bef3f846ac9be9d55adcf17497a not found: ID does not exist"
Sep 19 06:01:10.477650 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.477613    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="554e22bd-2655-4605-b41e-192d357145a4" path="/var/lib/kubelet/pods/554e22bd-2655-4605-b41e-192d357145a4/volumes"
Sep 19 06:01:10.556386 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556351    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-home\") pod \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") "
Sep 19 06:01:10.556593 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556441    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-model-cache\") pod \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") "
Sep 19 06:01:10.556593 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556468    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tmp-dir\") pod \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") "
Sep 19 06:01:10.556593 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556508    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-dshm\") pod \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") "
Sep 19 06:01:10.556593 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tls-certs\") pod \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") "
Sep 19 06:01:10.556593 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556579    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwxvg\" (UniqueName: \"kubernetes.io/projected/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-kube-api-access-mwxvg\") pod \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\" (UID: \"8f5ba7af-9e13-4f33-81a6-0bbe252c21e6\") "
Sep 19 06:01:10.556849 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556711    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-home" (OuterVolumeSpecName: "home") pod "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" (UID: "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:10.556849 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556756    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" (UID: "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:10.556849 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556769    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" (UID: "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:10.557006 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.556985    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.557081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.557010    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.557081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.557066    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.559072 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.559026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-kube-api-access-mwxvg" (OuterVolumeSpecName: "kube-api-access-mwxvg") pod "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" (UID: "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6"). InnerVolumeSpecName "kube-api-access-mwxvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:10.559072 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.559053    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-dshm" (OuterVolumeSpecName: "dshm") pod "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" (UID: "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:10.559072 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.559049    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" (UID: "8f5ba7af-9e13-4f33-81a6-0bbe252c21e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:10.658756 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.658667    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.658756 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.658698    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.658756 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.658709    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwxvg\" (UniqueName: \"kubernetes.io/projected/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6-kube-api-access-mwxvg\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:10.751213 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.751183    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"]
Sep 19 06:01:10.757478 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:10.757454    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c85316/test-model-c85316-kserve-9d84bc597-xxmlm"]
Sep 19 06:01:12.477583 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:12.477550    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" path="/var/lib/kubelet/pods/8f5ba7af-9e13-4f33-81a6-0bbe252c21e6/volumes"
Sep 19 06:01:25.172175 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172132    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"]
Sep 19 06:01:25.172642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172535    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" containerName="main"
Sep 19 06:01:25.172642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172549    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" containerName="main"
Sep 19 06:01:25.172642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172566    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554e22bd-2655-4605-b41e-192d357145a4" containerName="payload-pre-processing"
Sep 19 06:01:25.172642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172571    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="554e22bd-2655-4605-b41e-192d357145a4" containerName="payload-pre-processing"
Sep 19 06:01:25.172642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172634    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="554e22bd-2655-4605-b41e-192d357145a4" containerName="payload-pre-processing"
Sep 19 06:01:25.172642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.172641    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f5ba7af-9e13-4f33-81a6-0bbe252c21e6" containerName="main"
Sep 19 06:01:25.175831 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.175814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.178612 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.178595    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-888f75\""
Sep 19 06:01:25.185714 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.185688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"]
Sep 19 06:01:25.195106 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.195078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq9n7\" (UniqueName: \"kubernetes.io/projected/1723097d-ff40-4ea0-aaae-c92ee5d747db-kube-api-access-zq9n7\") pod \"payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.195266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.195119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1723097d-ff40-4ea0-aaae-c92ee5d747db-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.296304 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.296242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq9n7\" (UniqueName: \"kubernetes.io/projected/1723097d-ff40-4ea0-aaae-c92ee5d747db-kube-api-access-zq9n7\") pod \"payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.296502 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.296312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1723097d-ff40-4ea0-aaae-c92ee5d747db-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.297321 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.297282    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1723097d-ff40-4ea0-aaae-c92ee5d747db-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.305522 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.305500    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq9n7\" (UniqueName: \"kubernetes.io/projected/1723097d-ff40-4ea0-aaae-c92ee5d747db-kube-api-access-zq9n7\") pod \"payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.487811 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.487715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:25.760293 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:25.760263    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"]
Sep 19 06:01:25.762155 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:01:25.762129    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1723097d_ff40_4ea0_aaae_c92ee5d747db.slice/crio-adebc85ce62b8cb7341ad865e26af51d7589e55eeefd0eb45feceb2fb42ca9ad WatchSource:0}: Error finding container adebc85ce62b8cb7341ad865e26af51d7589e55eeefd0eb45feceb2fb42ca9ad: Status 404 returned error can't find the container with id adebc85ce62b8cb7341ad865e26af51d7589e55eeefd0eb45feceb2fb42ca9ad
Sep 19 06:01:26.519899 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:26.519862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9" event={"ID":"1723097d-ff40-4ea0-aaae-c92ee5d747db","Type":"ContainerStarted","Data":"cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df"}
Sep 19 06:01:26.520413 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:26.519919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9" event={"ID":"1723097d-ff40-4ea0-aaae-c92ee5d747db","Type":"ContainerStarted","Data":"adebc85ce62b8cb7341ad865e26af51d7589e55eeefd0eb45feceb2fb42ca9ad"}
Sep 19 06:01:26.520413 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:26.519970    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:26.540003 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:26.539946    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9" podStartSLOduration=1.539925336 podStartE2EDuration="1.539925336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:26.538245815 +0000 UTC m=+1232.148925924" watchObservedRunningTime="2026-09-19 06:01:26.539925336 +0000 UTC m=+1232.150605300"
Sep 19 06:01:34.061300 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:34.061259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:01:36.588008 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:36.587929    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"]
Sep 19 06:01:37.537002 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:37.536976    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:01:48.579819 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.579786    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"]
Sep 19 06:01:48.580338 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.580068    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk" podUID="f1f8efde-3b9a-4119-b0e7-726792243e47" containerName="maas-api" containerID="cri-o://4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab" gracePeriod=30
Sep 19 06:01:48.611167 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.611141    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"]
Sep 19 06:01:48.613870 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.613846    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-cc0e26-29829960-d8dss"]
Sep 19 06:01:48.836109 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.836026    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 06:01:48.926065 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.926006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-metrics-certs\") pod \"f1f8efde-3b9a-4119-b0e7-726792243e47\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") "
Sep 19 06:01:48.926250 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.926144    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f8efde-3b9a-4119-b0e7-726792243e47-openshift-service-ca\") pod \"f1f8efde-3b9a-4119-b0e7-726792243e47\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") "
Sep 19 06:01:48.926250 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.926223    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zxsq\" (UniqueName: \"kubernetes.io/projected/f1f8efde-3b9a-4119-b0e7-726792243e47-kube-api-access-4zxsq\") pod \"f1f8efde-3b9a-4119-b0e7-726792243e47\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") "
Sep 19 06:01:48.926367 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.926248    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-maas-api-tls\") pod \"f1f8efde-3b9a-4119-b0e7-726792243e47\" (UID: \"f1f8efde-3b9a-4119-b0e7-726792243e47\") "
Sep 19 06:01:48.926519 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.926492    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f8efde-3b9a-4119-b0e7-726792243e47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1f8efde-3b9a-4119-b0e7-726792243e47" (UID: "f1f8efde-3b9a-4119-b0e7-726792243e47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:01:48.928443 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.928421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f1f8efde-3b9a-4119-b0e7-726792243e47" (UID: "f1f8efde-3b9a-4119-b0e7-726792243e47"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:48.928443 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.928434    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1f8efde-3b9a-4119-b0e7-726792243e47" (UID: "f1f8efde-3b9a-4119-b0e7-726792243e47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:48.928569 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:48.928487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f8efde-3b9a-4119-b0e7-726792243e47-kube-api-access-4zxsq" (OuterVolumeSpecName: "kube-api-access-4zxsq") pod "f1f8efde-3b9a-4119-b0e7-726792243e47" (UID: "f1f8efde-3b9a-4119-b0e7-726792243e47"). InnerVolumeSpecName "kube-api-access-4zxsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:49.027524 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.027488    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zxsq\" (UniqueName: \"kubernetes.io/projected/f1f8efde-3b9a-4119-b0e7-726792243e47-kube-api-access-4zxsq\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:49.027524 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.027520    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:49.027524 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.027529    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1f8efde-3b9a-4119-b0e7-726792243e47-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:49.027757 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.027538    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f8efde-3b9a-4119-b0e7-726792243e47-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:49.657527 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.657485    2581 generic.go:356] "Generic (PLEG): container finished" podID="f1f8efde-3b9a-4119-b0e7-726792243e47" containerID="4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab" exitCode=0
Sep 19 06:01:49.657938 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.657529    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"
Sep 19 06:01:49.657938 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.657536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk" event={"ID":"f1f8efde-3b9a-4119-b0e7-726792243e47","Type":"ContainerDied","Data":"4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab"}
Sep 19 06:01:49.657938 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.657599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk" event={"ID":"f1f8efde-3b9a-4119-b0e7-726792243e47","Type":"ContainerDied","Data":"d01016ff9c2e0cc866ad56b5eaba402188d5f21b63d8a6ac24b6998252716746"}
Sep 19 06:01:49.657938 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.657628    2581 scope.go:117] "RemoveContainer" containerID="4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab"
Sep 19 06:01:49.667917 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.667897    2581 scope.go:117] "RemoveContainer" containerID="4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab"
Sep 19 06:01:49.668217 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:01:49.668199    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab\": container with ID starting with 4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab not found: ID does not exist" containerID="4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab"
Sep 19 06:01:49.668280 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.668228    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab"} err="failed to get container status \"4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab\": rpc error: code = NotFound desc = could not find container \"4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab\": container with ID starting with 4065f75807dc3b096d268f190564d15a774973174ac7088bb9a83e2dabe101ab not found: ID does not exist"
Sep 19 06:01:49.679149 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.679124    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"]
Sep 19 06:01:49.684943 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.684921    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc0e26-69fc58db-x7zhk"]
Sep 19 06:01:49.780920 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.780889    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"]
Sep 19 06:01:49.781317 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.781304    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f8efde-3b9a-4119-b0e7-726792243e47" containerName="maas-api"
Sep 19 06:01:49.781368 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.781320    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f8efde-3b9a-4119-b0e7-726792243e47" containerName="maas-api"
Sep 19 06:01:49.781418 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.781408    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f8efde-3b9a-4119-b0e7-726792243e47" containerName="maas-api"
Sep 19 06:01:49.785935 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.785914    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:49.789076 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.789055    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-f1aad1-serving-cert\""
Sep 19 06:01:49.806284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.806261    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"]
Sep 19 06:01:49.937165 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.937082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61bf6ba4-212a-4d32-a744-fbc4b032cceb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:49.937165 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.937126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:49.937353 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.937202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:49.937353 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:49.937230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn6ln\" (UniqueName: \"kubernetes.io/projected/61bf6ba4-212a-4d32-a744-fbc4b032cceb-kube-api-access-bn6ln\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.038230 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.038201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61bf6ba4-212a-4d32-a744-fbc4b032cceb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.038396 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.038243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.038396 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.038279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.038396 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.038306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn6ln\" (UniqueName: \"kubernetes.io/projected/61bf6ba4-212a-4d32-a744-fbc4b032cceb-kube-api-access-bn6ln\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.038986 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.038901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61bf6ba4-212a-4d32-a744-fbc4b032cceb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.040915 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.040891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.041013 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.040939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.048642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.048618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn6ln\" (UniqueName: \"kubernetes.io/projected/61bf6ba4-212a-4d32-a744-fbc4b032cceb-kube-api-access-bn6ln\") pod \"maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.099444 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.099410    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:50.375613 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.375585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"]
Sep 19 06:01:50.378137 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:01:50.378015    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61bf6ba4_212a_4d32_a744_fbc4b032cceb.slice/crio-ae2335acb933c52b3ca5484bc20b059109db232d59f463c382185295709e09f2 WatchSource:0}: Error finding container ae2335acb933c52b3ca5484bc20b059109db232d59f463c382185295709e09f2: Status 404 returned error can't find the container with id ae2335acb933c52b3ca5484bc20b059109db232d59f463c382185295709e09f2
Sep 19 06:01:50.477485 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.477455    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a15535-402c-4e26-9ce2-b991dc2c8ced" path="/var/lib/kubelet/pods/52a15535-402c-4e26-9ce2-b991dc2c8ced/volumes"
Sep 19 06:01:50.477809 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.477797    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f8efde-3b9a-4119-b0e7-726792243e47" path="/var/lib/kubelet/pods/f1f8efde-3b9a-4119-b0e7-726792243e47/volumes"
Sep 19 06:01:50.663734 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:50.663647    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft" event={"ID":"61bf6ba4-212a-4d32-a744-fbc4b032cceb","Type":"ContainerStarted","Data":"ae2335acb933c52b3ca5484bc20b059109db232d59f463c382185295709e09f2"}
Sep 19 06:01:51.608442 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.608398    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm" podUID="c6546b48-1549-41e5-ad72-19ea7c3781d4" containerName="main" containerID="cri-o://f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387" gracePeriod=45
Sep 19 06:01:51.671799 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.671765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft" event={"ID":"61bf6ba4-212a-4d32-a744-fbc4b032cceb","Type":"ContainerStarted","Data":"6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2"}
Sep 19 06:01:51.672183 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.671904    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:01:51.691808 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.691749    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft" podStartSLOduration=2.216527056 podStartE2EDuration="2.691730586s" totalImagesPullingTime="475.20353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:51.691695726 +0000 UTC m=+1257.302375724" watchObservedRunningTime="2026-09-19 06:01:51.691730586 +0000 UTC m=+1257.302410577"
Sep 19 06:01:51.863015 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.862958    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 06:01:51.957356 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957315    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-dshm\") pod \"c6546b48-1549-41e5-ad72-19ea7c3781d4\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") "
Sep 19 06:01:51.957356 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957352    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-home\") pod \"c6546b48-1549-41e5-ad72-19ea7c3781d4\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") "
Sep 19 06:01:51.957592 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957397    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6546b48-1549-41e5-ad72-19ea7c3781d4-tls-certs\") pod \"c6546b48-1549-41e5-ad72-19ea7c3781d4\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") "
Sep 19 06:01:51.957592 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-model-cache\") pod \"c6546b48-1549-41e5-ad72-19ea7c3781d4\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") "
Sep 19 06:01:51.957592 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-tmp-dir\") pod \"c6546b48-1549-41e5-ad72-19ea7c3781d4\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") "
Sep 19 06:01:51.957592 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957536    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skwnd\" (UniqueName: \"kubernetes.io/projected/c6546b48-1549-41e5-ad72-19ea7c3781d4-kube-api-access-skwnd\") pod \"c6546b48-1549-41e5-ad72-19ea7c3781d4\" (UID: \"c6546b48-1549-41e5-ad72-19ea7c3781d4\") "
Sep 19 06:01:51.957804 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-home" (OuterVolumeSpecName: "home") pod "c6546b48-1549-41e5-ad72-19ea7c3781d4" (UID: "c6546b48-1549-41e5-ad72-19ea7c3781d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.957804 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957783    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6546b48-1549-41e5-ad72-19ea7c3781d4" (UID: "c6546b48-1549-41e5-ad72-19ea7c3781d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.957917 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6546b48-1549-41e5-ad72-19ea7c3781d4" (UID: "c6546b48-1549-41e5-ad72-19ea7c3781d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.957980 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957915    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.957980 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957932    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.957980 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.957948    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.959771 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.959744    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-dshm" (OuterVolumeSpecName: "dshm") pod "c6546b48-1549-41e5-ad72-19ea7c3781d4" (UID: "c6546b48-1549-41e5-ad72-19ea7c3781d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.959908 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.959777    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6546b48-1549-41e5-ad72-19ea7c3781d4-kube-api-access-skwnd" (OuterVolumeSpecName: "kube-api-access-skwnd") pod "c6546b48-1549-41e5-ad72-19ea7c3781d4" (UID: "c6546b48-1549-41e5-ad72-19ea7c3781d4"). InnerVolumeSpecName "kube-api-access-skwnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:51.959908 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:51.959871    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6546b48-1549-41e5-ad72-19ea7c3781d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6546b48-1549-41e5-ad72-19ea7c3781d4" (UID: "c6546b48-1549-41e5-ad72-19ea7c3781d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:52.058839 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.058808    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6546b48-1549-41e5-ad72-19ea7c3781d4-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.058839 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.058833    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6546b48-1549-41e5-ad72-19ea7c3781d4-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.058839 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.058843    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skwnd\" (UniqueName: \"kubernetes.io/projected/c6546b48-1549-41e5-ad72-19ea7c3781d4-kube-api-access-skwnd\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.639534 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.639496    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"]
Sep 19 06:01:52.639948 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.639929    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6546b48-1549-41e5-ad72-19ea7c3781d4" containerName="main"
Sep 19 06:01:52.639948 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.639946    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6546b48-1549-41e5-ad72-19ea7c3781d4" containerName="main"
Sep 19 06:01:52.640168 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.640058    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6546b48-1549-41e5-ad72-19ea7c3781d4" containerName="main"
Sep 19 06:01:52.643530 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.643513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.647587 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.647559    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-957d1190\"/\"e2e-lifecycle-model-957d1190-kserve-self-signed-certs\""
Sep 19 06:01:52.647695 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.647621    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-957d1190\"/\"kube-root-ca.crt\""
Sep 19 06:01:52.647695 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.647674    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-957d1190\"/\"default-dockercfg-rzn6j\""
Sep 19 06:01:52.647814 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.647621    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-957d1190\"/\"openshift-service-ca.crt\""
Sep 19 06:01:52.653242 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.653219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"]
Sep 19 06:01:52.678578 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.678551    2581 generic.go:356] "Generic (PLEG): container finished" podID="c6546b48-1549-41e5-ad72-19ea7c3781d4" containerID="f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387" exitCode=0
Sep 19 06:01:52.679025 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.678616    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"
Sep 19 06:01:52.679025 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.678634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm" event={"ID":"c6546b48-1549-41e5-ad72-19ea7c3781d4","Type":"ContainerDied","Data":"f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387"}
Sep 19 06:01:52.679025 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.678678    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm" event={"ID":"c6546b48-1549-41e5-ad72-19ea7c3781d4","Type":"ContainerDied","Data":"93b54fd177e5919e46d551e32343c2c12087fb57e5ee93a3a2fda92b93269110"}
Sep 19 06:01:52.679025 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.678700    2581 scope.go:117] "RemoveContainer" containerID="f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387"
Sep 19 06:01:52.688149 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.688115    2581 scope.go:117] "RemoveContainer" containerID="f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387"
Sep 19 06:01:52.688448 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:01:52.688423    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387\": container with ID starting with f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387 not found: ID does not exist" containerID="f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387"
Sep 19 06:01:52.688558 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.688457    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387"} err="failed to get container status \"f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387\": rpc error: code = NotFound desc = could not find container \"f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387\": container with ID starting with f14cf852f3ecad126f77794ce4b271aab48372a4ed0f9080227cc0426bbce387 not found: ID does not exist"
Sep 19 06:01:52.697443 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.697416    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"]
Sep 19 06:01:52.704465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.704439    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/simulator-w0-cc0e26-kserve-5b9cc68db-l6hcm"]
Sep 19 06:01:52.766393 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.766315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-dshm\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.766393 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.766356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-tmp-dir\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.766393 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.766388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp55l\" (UniqueName: \"kubernetes.io/projected/f1111983-4d2f-4812-9e90-85d342079523-kube-api-access-rp55l\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.766629 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.766444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-home\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.766629 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.766493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-model-cache\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.766629 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.766565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1111983-4d2f-4812-9e90-85d342079523-tls-certs\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.867576 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.867537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1111983-4d2f-4812-9e90-85d342079523-tls-certs\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.867748 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.867625    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-dshm\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.867748 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.867691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-tmp-dir\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.867748 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.867739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp55l\" (UniqueName: \"kubernetes.io/projected/f1111983-4d2f-4812-9e90-85d342079523-kube-api-access-rp55l\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.867924 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.867765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-home\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.867924 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.867803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-model-cache\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.868190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.868168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-home\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.868285 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.868196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-tmp-dir\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.868285 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.868219    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-model-cache\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.870367 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.870347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1111983-4d2f-4812-9e90-85d342079523-tls-certs\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.870427 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.870386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-dshm\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.876840 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.876822    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp55l\" (UniqueName: \"kubernetes.io/projected/f1111983-4d2f-4812-9e90-85d342079523-kube-api-access-rp55l\") pod \"e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") " pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:52.954452 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:52.954423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:01:53.237171 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:53.237146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"]
Sep 19 06:01:53.238937 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:01:53.238911    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1111983_4d2f_4812_9e90_85d342079523.slice/crio-c76a4ad4d4b31767e42b26f7781f6917839ce6597115fade63969a1d1dde3fed WatchSource:0}: Error finding container c76a4ad4d4b31767e42b26f7781f6917839ce6597115fade63969a1d1dde3fed: Status 404 returned error can't find the container with id c76a4ad4d4b31767e42b26f7781f6917839ce6597115fade63969a1d1dde3fed
Sep 19 06:01:53.687869 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:53.687824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd" event={"ID":"f1111983-4d2f-4812-9e90-85d342079523","Type":"ContainerStarted","Data":"c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2"}
Sep 19 06:01:53.688322 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:53.687879    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd" event={"ID":"f1111983-4d2f-4812-9e90-85d342079523","Type":"ContainerStarted","Data":"c76a4ad4d4b31767e42b26f7781f6917839ce6597115fade63969a1d1dde3fed"}
Sep 19 06:01:53.708246 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:53.708195    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd" podStartSLOduration=1.7081790479999999 podStartE2EDuration="1.708179048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:53.705977487 +0000 UTC m=+1259.316657468" watchObservedRunningTime="2026-09-19 06:01:53.708179048 +0000 UTC m=+1259.318859025"
Sep 19 06:01:54.479293 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:54.479259    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6546b48-1549-41e5-ad72-19ea7c3781d4" path="/var/lib/kubelet/pods/c6546b48-1549-41e5-ad72-19ea7c3781d4/volumes"
Sep 19 06:01:57.684496 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:01:57.684468    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:02:02.955133 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:02.955097    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:02:02.955524 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:02.955182    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:02:02.967785 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:02.967763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:02:03.753015 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:03.752984    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:02:04.553557 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:04.553523    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:02:34.545306 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:34.545265    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:02:39.448285 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.448252    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"]
Sep 19 06:02:39.452194 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.452172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.454831 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.454809    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"default-dockercfg-v9psp\""
Sep 19 06:02:39.454982 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.454849    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"kube-root-ca.crt\""
Sep 19 06:02:39.456148 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.456097    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"openshift-service-ca.crt\""
Sep 19 06:02:39.456824 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.456795    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"auth-test-model-888f75-kserve-self-signed-certs\""
Sep 19 06:02:39.459075 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.459054    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"]
Sep 19 06:02:39.586424 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.586383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-dshm\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.586580 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.586435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-home\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.586580 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.586523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-model-cache\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.586685 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.586592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85487\" (UniqueName: \"kubernetes.io/projected/309764c0-3a4a-45e5-adbe-63a5eec2cc74-kube-api-access-85487\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.586685 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.586638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tmp-dir\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.586685 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.586667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tls-certs\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.687742 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.687706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85487\" (UniqueName: \"kubernetes.io/projected/309764c0-3a4a-45e5-adbe-63a5eec2cc74-kube-api-access-85487\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.687951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.687758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tmp-dir\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.687951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.687777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tls-certs\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.687951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.687814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-dshm\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.687951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.687844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-home\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.687951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.687878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-model-cache\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.688283 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.688229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tmp-dir\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.688340 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.688301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-home\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.688400 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.688390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-model-cache\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.690327 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.690306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-dshm\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.690772 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.690751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tls-certs\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.696051 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.696014    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85487\" (UniqueName: \"kubernetes.io/projected/309764c0-3a4a-45e5-adbe-63a5eec2cc74-kube-api-access-85487\") pod \"auth-test-model-888f75-kserve-676c44595f-4d4rh\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") " pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:39.764834 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:39.764774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:40.039752 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:40.039722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"]
Sep 19 06:02:40.041139 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:02:40.041109    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod309764c0_3a4a_45e5_adbe_63a5eec2cc74.slice/crio-350cfcf83dcdbb6e445631b7ad31fc4f7d404ca0d4719ebd5c812543b297c341 WatchSource:0}: Error finding container 350cfcf83dcdbb6e445631b7ad31fc4f7d404ca0d4719ebd5c812543b297c341: Status 404 returned error can't find the container with id 350cfcf83dcdbb6e445631b7ad31fc4f7d404ca0d4719ebd5c812543b297c341
Sep 19 06:02:40.945105 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:40.945075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh" event={"ID":"309764c0-3a4a-45e5-adbe-63a5eec2cc74","Type":"ContainerStarted","Data":"4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c"}
Sep 19 06:02:40.945503 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:40.945115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh" event={"ID":"309764c0-3a4a-45e5-adbe-63a5eec2cc74","Type":"ContainerStarted","Data":"350cfcf83dcdbb6e445631b7ad31fc4f7d404ca0d4719ebd5c812543b297c341"}
Sep 19 06:02:40.966014 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:40.965968    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh" podStartSLOduration=1.9659525960000002 podStartE2EDuration="1.965952596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:02:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:02:40.964633904 +0000 UTC m=+1306.575313897" watchObservedRunningTime="2026-09-19 06:02:40.965952596 +0000 UTC m=+1306.576632560"
Sep 19 06:02:42.440406 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:42.440373    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"]
Sep 19 06:02:49.765387 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:49.765350    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:49.765964 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:49.765556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:49.778054 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:49.778017    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:50.006670 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:50.006645    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:02:56.290968 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.290932    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"]
Sep 19 06:02:56.294491 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.294431    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b" podUID="3103d793-b0be-4076-a029-6a476dd1ba20" containerName="payload-pre-processing" containerID="cri-o://9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3" gracePeriod=30
Sep 19 06:02:56.548361 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.548304    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:02:56.639168 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.639138    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8f9m\" (UniqueName: \"kubernetes.io/projected/3103d793-b0be-4076-a029-6a476dd1ba20-kube-api-access-m8f9m\") pod \"3103d793-b0be-4076-a029-6a476dd1ba20\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") "
Sep 19 06:02:56.639324 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.639217    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3103d793-b0be-4076-a029-6a476dd1ba20-plugins-config-volume\") pod \"3103d793-b0be-4076-a029-6a476dd1ba20\" (UID: \"3103d793-b0be-4076-a029-6a476dd1ba20\") "
Sep 19 06:02:56.639596 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.639573    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3103d793-b0be-4076-a029-6a476dd1ba20-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3103d793-b0be-4076-a029-6a476dd1ba20" (UID: "3103d793-b0be-4076-a029-6a476dd1ba20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:02:56.641318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.641296    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3103d793-b0be-4076-a029-6a476dd1ba20-kube-api-access-m8f9m" (OuterVolumeSpecName: "kube-api-access-m8f9m") pod "3103d793-b0be-4076-a029-6a476dd1ba20" (UID: "3103d793-b0be-4076-a029-6a476dd1ba20"). InnerVolumeSpecName "kube-api-access-m8f9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:02:56.739897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.739866    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3103d793-b0be-4076-a029-6a476dd1ba20-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:56.739897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:56.739891    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8f9m\" (UniqueName: \"kubernetes.io/projected/3103d793-b0be-4076-a029-6a476dd1ba20-kube-api-access-m8f9m\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:57.033921 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.033890    2581 generic.go:356] "Generic (PLEG): container finished" podID="3103d793-b0be-4076-a029-6a476dd1ba20" containerID="9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3" exitCode=0
Sep 19 06:02:57.034115 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.033929    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"
Sep 19 06:02:57.034115 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.033969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b" event={"ID":"3103d793-b0be-4076-a029-6a476dd1ba20","Type":"ContainerDied","Data":"9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3"}
Sep 19 06:02:57.034115 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.034008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b" event={"ID":"3103d793-b0be-4076-a029-6a476dd1ba20","Type":"ContainerDied","Data":"9e09983cae5d5e49e1f68e3f0d87d1f8f7166f8cd007fd71e10e3e17a25cf338"}
Sep 19 06:02:57.034115 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.034028    2581 scope.go:117] "RemoveContainer" containerID="9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3"
Sep 19 06:02:57.043282 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.043261    2581 scope.go:117] "RemoveContainer" containerID="9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3"
Sep 19 06:02:57.043552 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:02:57.043533    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3\": container with ID starting with 9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3 not found: ID does not exist" containerID="9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3"
Sep 19 06:02:57.043615 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.043562    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3"} err="failed to get container status \"9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3\": rpc error: code = NotFound desc = could not find container \"9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3\": container with ID starting with 9d0f07b863249cba623b9a097cf9b62428bb83479e44ad0ea9cc9d961cb915a3 not found: ID does not exist"
Sep 19 06:02:57.056460 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.056436    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"]
Sep 19 06:02:57.060102 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.060068    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-957d1190-59d5f7df6b-99q2b"]
Sep 19 06:02:57.460650 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.460557    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv" podUID="ae388a09-e072-4fd7-ade6-3011f5fd113b" containerName="main" containerID="cri-o://ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa" gracePeriod=45
Sep 19 06:02:57.716066 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.715997    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 06:02:57.750204 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-dshm\") pod \"ae388a09-e072-4fd7-ade6-3011f5fd113b\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") "
Sep 19 06:02:57.750332 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750233    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-home\") pod \"ae388a09-e072-4fd7-ade6-3011f5fd113b\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") "
Sep 19 06:02:57.750332 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-tmp-dir\") pod \"ae388a09-e072-4fd7-ade6-3011f5fd113b\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") "
Sep 19 06:02:57.750455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750341    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qtkg\" (UniqueName: \"kubernetes.io/projected/ae388a09-e072-4fd7-ade6-3011f5fd113b-kube-api-access-5qtkg\") pod \"ae388a09-e072-4fd7-ade6-3011f5fd113b\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") "
Sep 19 06:02:57.750455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750374    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae388a09-e072-4fd7-ade6-3011f5fd113b-tls-certs\") pod \"ae388a09-e072-4fd7-ade6-3011f5fd113b\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") "
Sep 19 06:02:57.750455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-model-cache\") pod \"ae388a09-e072-4fd7-ade6-3011f5fd113b\" (UID: \"ae388a09-e072-4fd7-ade6-3011f5fd113b\") "
Sep 19 06:02:57.750605 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750504    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-home" (OuterVolumeSpecName: "home") pod "ae388a09-e072-4fd7-ade6-3011f5fd113b" (UID: "ae388a09-e072-4fd7-ade6-3011f5fd113b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:57.750605 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750565    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ae388a09-e072-4fd7-ade6-3011f5fd113b" (UID: "ae388a09-e072-4fd7-ade6-3011f5fd113b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:57.750796 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750776    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-model-cache" (OuterVolumeSpecName: "model-cache") pod "ae388a09-e072-4fd7-ade6-3011f5fd113b" (UID: "ae388a09-e072-4fd7-ade6-3011f5fd113b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:57.751000 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.750914    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:57.751091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.751057    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:57.752554 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.752525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-dshm" (OuterVolumeSpecName: "dshm") pod "ae388a09-e072-4fd7-ade6-3011f5fd113b" (UID: "ae388a09-e072-4fd7-ade6-3011f5fd113b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:57.753073 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.753025    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae388a09-e072-4fd7-ade6-3011f5fd113b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae388a09-e072-4fd7-ade6-3011f5fd113b" (UID: "ae388a09-e072-4fd7-ade6-3011f5fd113b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:02:57.753150 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.753099    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae388a09-e072-4fd7-ade6-3011f5fd113b-kube-api-access-5qtkg" (OuterVolumeSpecName: "kube-api-access-5qtkg") pod "ae388a09-e072-4fd7-ade6-3011f5fd113b" (UID: "ae388a09-e072-4fd7-ade6-3011f5fd113b"). InnerVolumeSpecName "kube-api-access-5qtkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:02:57.851691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.851663    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qtkg\" (UniqueName: \"kubernetes.io/projected/ae388a09-e072-4fd7-ade6-3011f5fd113b-kube-api-access-5qtkg\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:57.851691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.851688    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae388a09-e072-4fd7-ade6-3011f5fd113b-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:57.851691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.851697    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:57.851894 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:57.851704    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae388a09-e072-4fd7-ade6-3011f5fd113b-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:02:58.040989 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.040962    2581 generic.go:356] "Generic (PLEG): container finished" podID="ae388a09-e072-4fd7-ade6-3011f5fd113b" containerID="ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa" exitCode=0
Sep 19 06:02:58.041180 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.041006    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"
Sep 19 06:02:58.041180 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.041067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv" event={"ID":"ae388a09-e072-4fd7-ade6-3011f5fd113b","Type":"ContainerDied","Data":"ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa"}
Sep 19 06:02:58.041180 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.041105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv" event={"ID":"ae388a09-e072-4fd7-ade6-3011f5fd113b","Type":"ContainerDied","Data":"7cc06910b71167a904d7c2af13957486e25c80fdced41ca5ebcbf04483f81197"}
Sep 19 06:02:58.041180 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.041127    2581 scope.go:117] "RemoveContainer" containerID="ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa"
Sep 19 06:02:58.052975 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.052932    2581 scope.go:117] "RemoveContainer" containerID="ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa"
Sep 19 06:02:58.053235 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:02:58.053209    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa\": container with ID starting with ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa not found: ID does not exist" containerID="ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa"
Sep 19 06:02:58.053347 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.053244    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa"} err="failed to get container status \"ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa\": rpc error: code = NotFound desc = could not find container \"ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa\": container with ID starting with ac248826b21bdc233b351be2c9db1a4d1bc1cda842885fa34520473857543bfa not found: ID does not exist"
Sep 19 06:02:58.062689 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.062668    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"]
Sep 19 06:02:58.066185 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.066163    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-a-95222d07-kserve-79955bfcfd-lwwgv"]
Sep 19 06:02:58.478088 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.477985    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3103d793-b0be-4076-a029-6a476dd1ba20" path="/var/lib/kubelet/pods/3103d793-b0be-4076-a029-6a476dd1ba20/volumes"
Sep 19 06:02:58.478473 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:02:58.478457    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae388a09-e072-4fd7-ade6-3011f5fd113b" path="/var/lib/kubelet/pods/ae388a09-e072-4fd7-ade6-3011f5fd113b/volumes"
Sep 19 06:03:03.242779 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:03.242741    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:03:04.683968 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:04.683935    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"]
Sep 19 06:03:19.704424 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:19.704382    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd" podUID="f1111983-4d2f-4812-9e90-85d342079523" containerName="main" containerID="cri-o://c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2" gracePeriod=45
Sep 19 06:03:19.968365 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:19.968308    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:03:20.050899 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.050866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-home\") pod \"f1111983-4d2f-4812-9e90-85d342079523\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") "
Sep 19 06:03:20.051101 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.050924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-model-cache\") pod \"f1111983-4d2f-4812-9e90-85d342079523\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") "
Sep 19 06:03:20.051101 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.050956    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp55l\" (UniqueName: \"kubernetes.io/projected/f1111983-4d2f-4812-9e90-85d342079523-kube-api-access-rp55l\") pod \"f1111983-4d2f-4812-9e90-85d342079523\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") "
Sep 19 06:03:20.051101 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.050980    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1111983-4d2f-4812-9e90-85d342079523-tls-certs\") pod \"f1111983-4d2f-4812-9e90-85d342079523\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") "
Sep 19 06:03:20.051101 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051011    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-dshm\") pod \"f1111983-4d2f-4812-9e90-85d342079523\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") "
Sep 19 06:03:20.051327 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051164    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-home" (OuterVolumeSpecName: "home") pod "f1111983-4d2f-4812-9e90-85d342079523" (UID: "f1111983-4d2f-4812-9e90-85d342079523"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:03:20.051327 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1111983-4d2f-4812-9e90-85d342079523" (UID: "f1111983-4d2f-4812-9e90-85d342079523"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:03:20.051327 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051296    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-tmp-dir\") pod \"f1111983-4d2f-4812-9e90-85d342079523\" (UID: \"f1111983-4d2f-4812-9e90-85d342079523\") "
Sep 19 06:03:20.051567 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1111983-4d2f-4812-9e90-85d342079523" (UID: "f1111983-4d2f-4812-9e90-85d342079523"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:03:20.051760 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051742    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:03:20.051836 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051765    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:03:20.051836 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.051780    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:03:20.053316 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.053297    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-dshm" (OuterVolumeSpecName: "dshm") pod "f1111983-4d2f-4812-9e90-85d342079523" (UID: "f1111983-4d2f-4812-9e90-85d342079523"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:03:20.053379 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.053340    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1111983-4d2f-4812-9e90-85d342079523-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1111983-4d2f-4812-9e90-85d342079523" (UID: "f1111983-4d2f-4812-9e90-85d342079523"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:03:20.053441 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.053424    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1111983-4d2f-4812-9e90-85d342079523-kube-api-access-rp55l" (OuterVolumeSpecName: "kube-api-access-rp55l") pod "f1111983-4d2f-4812-9e90-85d342079523" (UID: "f1111983-4d2f-4812-9e90-85d342079523"). InnerVolumeSpecName "kube-api-access-rp55l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:03:20.152568 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.152521    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp55l\" (UniqueName: \"kubernetes.io/projected/f1111983-4d2f-4812-9e90-85d342079523-kube-api-access-rp55l\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:03:20.152568 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.152561    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1111983-4d2f-4812-9e90-85d342079523-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:03:20.152568 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.152574    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1111983-4d2f-4812-9e90-85d342079523-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:03:20.163259 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.163232    2581 generic.go:356] "Generic (PLEG): container finished" podID="f1111983-4d2f-4812-9e90-85d342079523" containerID="c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2" exitCode=0
Sep 19 06:03:20.163376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.163274    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"
Sep 19 06:03:20.163376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.163288    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd" event={"ID":"f1111983-4d2f-4812-9e90-85d342079523","Type":"ContainerDied","Data":"c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2"}
Sep 19 06:03:20.163376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.163326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd" event={"ID":"f1111983-4d2f-4812-9e90-85d342079523","Type":"ContainerDied","Data":"c76a4ad4d4b31767e42b26f7781f6917839ce6597115fade63969a1d1dde3fed"}
Sep 19 06:03:20.163376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.163344    2581 scope.go:117] "RemoveContainer" containerID="c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2"
Sep 19 06:03:20.172946 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.172930    2581 scope.go:117] "RemoveContainer" containerID="c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2"
Sep 19 06:03:20.173243 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:03:20.173223    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2\": container with ID starting with c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2 not found: ID does not exist" containerID="c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2"
Sep 19 06:03:20.173323 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.173255    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2"} err="failed to get container status \"c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2\": rpc error: code = NotFound desc = could not find container \"c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2\": container with ID starting with c87b5b1eff9ea09ea1301e68fa657c1a6310550989e4410d1d71d9645bea1ae2 not found: ID does not exist"
Sep 19 06:03:20.184284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.184263    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"]
Sep 19 06:03:20.187279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.187258    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-957d1190/e2e-lifecycle-model-957d1190-kserve-6cbbf6f947-xh2wd"]
Sep 19 06:03:20.477238 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:20.477209    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1111983-4d2f-4812-9e90-85d342079523" path="/var/lib/kubelet/pods/f1111983-4d2f-4812-9e90-85d342079523/volumes"
Sep 19 06:03:24.249253 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249219    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"]
Sep 19 06:03:24.249843 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249818    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3103d793-b0be-4076-a029-6a476dd1ba20" containerName="payload-pre-processing"
Sep 19 06:03:24.249843 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249836    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3103d793-b0be-4076-a029-6a476dd1ba20" containerName="payload-pre-processing"
Sep 19 06:03:24.249843 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249843    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1111983-4d2f-4812-9e90-85d342079523" containerName="main"
Sep 19 06:03:24.249971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249850    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1111983-4d2f-4812-9e90-85d342079523" containerName="main"
Sep 19 06:03:24.249971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249861    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae388a09-e072-4fd7-ade6-3011f5fd113b" containerName="main"
Sep 19 06:03:24.249971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249866    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae388a09-e072-4fd7-ade6-3011f5fd113b" containerName="main"
Sep 19 06:03:24.249971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249943    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3103d793-b0be-4076-a029-6a476dd1ba20" containerName="payload-pre-processing"
Sep 19 06:03:24.249971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae388a09-e072-4fd7-ade6-3011f5fd113b" containerName="main"
Sep 19 06:03:24.249971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.249968    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1111983-4d2f-4812-9e90-85d342079523" containerName="main"
Sep 19 06:03:24.253443 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.253420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.256525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.256502    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"distinct-w4-3182b5-kserve-self-signed-certs\""
Sep 19 06:03:24.261753 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.261727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"]
Sep 19 06:03:24.387731 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.387697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-dshm\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.387896 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.387741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tmp-dir\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.387896 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.387786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tls-certs\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.387896 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.387836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-home\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.387896 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.387873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57kg\" (UniqueName: \"kubernetes.io/projected/3a83c1e8-77a4-4898-a2e9-89f4b6998554-kube-api-access-z57kg\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.388170 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.387906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-model-cache\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.488726 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.488697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-dshm\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.488927 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.488743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tmp-dir\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.488927 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.488760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tls-certs\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.488927 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.488801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-home\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.488927 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.488837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57kg\" (UniqueName: \"kubernetes.io/projected/3a83c1e8-77a4-4898-a2e9-89f4b6998554-kube-api-access-z57kg\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.488927 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.488867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-model-cache\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.489279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.489254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tmp-dir\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.489427 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.489262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-model-cache\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.489542 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.489513    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-home\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.491556 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.491535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tls-certs\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.491639 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.491620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-dshm\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.496861 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.496839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57kg\" (UniqueName: \"kubernetes.io/projected/3a83c1e8-77a4-4898-a2e9-89f4b6998554-kube-api-access-z57kg\") pod \"distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.563621 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.563596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:24.834700 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:24.834664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"]
Sep 19 06:03:24.835820 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:03:24.835788    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a83c1e8_77a4_4898_a2e9_89f4b6998554.slice/crio-4f2290c62a1da3900148d813ed24ab49ed2438c0b6798c2d7aa864715d6e2f0c WatchSource:0}: Error finding container 4f2290c62a1da3900148d813ed24ab49ed2438c0b6798c2d7aa864715d6e2f0c: Status 404 returned error can't find the container with id 4f2290c62a1da3900148d813ed24ab49ed2438c0b6798c2d7aa864715d6e2f0c
Sep 19 06:03:25.194440 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:25.194338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq" event={"ID":"3a83c1e8-77a4-4898-a2e9-89f4b6998554","Type":"ContainerStarted","Data":"c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c"}
Sep 19 06:03:25.194440 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:25.194395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq" event={"ID":"3a83c1e8-77a4-4898-a2e9-89f4b6998554","Type":"ContainerStarted","Data":"4f2290c62a1da3900148d813ed24ab49ed2438c0b6798c2d7aa864715d6e2f0c"}
Sep 19 06:03:25.212322 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:25.212277    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq" podStartSLOduration=1.212263735 podStartE2EDuration="1.212263735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:03:25.210548069 +0000 UTC m=+1350.821228061" watchObservedRunningTime="2026-09-19 06:03:25.212263735 +0000 UTC m=+1350.822943699"
Sep 19 06:03:28.753291 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:28.753260    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:03:34.564124 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:34.564085    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:34.564534 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:34.564140    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:34.576669 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:34.576645    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:03:35.260869 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:03:35.260836    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:04:00.452677 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:00.452642    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:04:20.446223 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:20.446188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:04:35.429951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.429917    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"]
Sep 19 06:04:35.433686 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.433668    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.436485 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.436458    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"distinct-2-w4-3182b5-kserve-self-signed-certs\""
Sep 19 06:04:35.443152 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.443120    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"]
Sep 19 06:04:35.553632 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.553599    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tmp-dir\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.553809 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.553688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tls-certs\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.553809 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.553746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-home\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.553809 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.553802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2g44\" (UniqueName: \"kubernetes.io/projected/11d48a1a-e73b-4675-bae1-96f9fcec9e95-kube-api-access-d2g44\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.553960 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.553832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-dshm\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.553960 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.553921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-model-cache\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.654613 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tmp-dir\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.654791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tls-certs\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.654791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-home\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.654791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654690    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2g44\" (UniqueName: \"kubernetes.io/projected/11d48a1a-e73b-4675-bae1-96f9fcec9e95-kube-api-access-d2g44\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.654791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-dshm\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.654791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-model-cache\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.655091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.654989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tmp-dir\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.655091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.655061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-home\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.655170 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.655098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-model-cache\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.657170 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.657141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-dshm\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.657304 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.657278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tls-certs\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.662081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.662058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2g44\" (UniqueName: \"kubernetes.io/projected/11d48a1a-e73b-4675-bae1-96f9fcec9e95-kube-api-access-d2g44\") pod \"distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") " pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:35.743520 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:35.743457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:36.014469 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:36.014437    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"]
Sep 19 06:04:36.016092 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:04:36.016025    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11d48a1a_e73b_4675_bae1_96f9fcec9e95.slice/crio-b39041fbd2983fa1386f873cecbfcd4d1dc3004ef8b120f2de4f696333567219 WatchSource:0}: Error finding container b39041fbd2983fa1386f873cecbfcd4d1dc3004ef8b120f2de4f696333567219: Status 404 returned error can't find the container with id b39041fbd2983fa1386f873cecbfcd4d1dc3004ef8b120f2de4f696333567219
Sep 19 06:04:36.571630 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:36.571554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j" event={"ID":"11d48a1a-e73b-4675-bae1-96f9fcec9e95","Type":"ContainerStarted","Data":"2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0"}
Sep 19 06:04:36.571630 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:36.571603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j" event={"ID":"11d48a1a-e73b-4675-bae1-96f9fcec9e95","Type":"ContainerStarted","Data":"b39041fbd2983fa1386f873cecbfcd4d1dc3004ef8b120f2de4f696333567219"}
Sep 19 06:04:36.588425 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:36.588371    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j" podStartSLOduration=1.5883525729999999 podStartE2EDuration="1.588352573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:04:36.587217722 +0000 UTC m=+1422.197897712" watchObservedRunningTime="2026-09-19 06:04:36.588352573 +0000 UTC m=+1422.199032537"
Sep 19 06:04:42.453845 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:42.453807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:04:45.743834 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:45.743801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:45.744270 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:45.743856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:45.756965 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:45.756942    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:04:46.638391 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:04:46.638358    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:05:12.341511 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:05:12.341470    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:05:40.541462 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:05:40.541424    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:05:56.505228 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:05:56.505201    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:05:56.511028 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:05:56.511005    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:05:56.511358 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:05:56.511334    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:05:56.516495 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:05:56.516479    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:06:16.045514 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:06:16.045476    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:06:44.645948 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:06:44.645912    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:06:56.660372 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:06:56.660336    2581 scope.go:117] "RemoveContainer" containerID="1f2172e731b779b1b142e702ee676bb84274e93c751daf97097c37482722625b"
Sep 19 06:07:10.679880 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:10.679831    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"]
Sep 19 06:07:10.685318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:10.685289    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"]
Sep 19 06:07:10.698753 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:10.698720    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"]
Sep 19 06:07:10.706996 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:10.706965    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"]
Sep 19 06:07:11.028524 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.028476    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"]
Sep 19 06:07:11.028724 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.028551    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2b3fb3-29829960-844j7"]
Sep 19 06:07:11.056374 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.056341    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"]
Sep 19 06:07:11.056670 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.056643    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq" podUID="69baeff6-8513-4b8f-9a61-9fa599e5f8b6" containerName="payload-pre-processing" containerID="cri-o://b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4" gracePeriod=30
Sep 19 06:07:11.311103 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.311076    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 06:07:11.341837 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.341783    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hcfz\" (UniqueName: \"kubernetes.io/projected/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-kube-api-access-5hcfz\") pod \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") "
Sep 19 06:07:11.341975 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.341942    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-plugins-config-volume\") pod \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\" (UID: \"69baeff6-8513-4b8f-9a61-9fa599e5f8b6\") "
Sep 19 06:07:11.342343 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.342314    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69baeff6-8513-4b8f-9a61-9fa599e5f8b6" (UID: "69baeff6-8513-4b8f-9a61-9fa599e5f8b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:11.344638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.344607    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-kube-api-access-5hcfz" (OuterVolumeSpecName: "kube-api-access-5hcfz") pod "69baeff6-8513-4b8f-9a61-9fa599e5f8b6" (UID: "69baeff6-8513-4b8f-9a61-9fa599e5f8b6"). InnerVolumeSpecName "kube-api-access-5hcfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:11.386163 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.386135    2581 generic.go:356] "Generic (PLEG): container finished" podID="69baeff6-8513-4b8f-9a61-9fa599e5f8b6" containerID="b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4" exitCode=0
Sep 19 06:07:11.386266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.386181    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"
Sep 19 06:07:11.386266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.386202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq" event={"ID":"69baeff6-8513-4b8f-9a61-9fa599e5f8b6","Type":"ContainerDied","Data":"b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4"}
Sep 19 06:07:11.386266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.386243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq" event={"ID":"69baeff6-8513-4b8f-9a61-9fa599e5f8b6","Type":"ContainerDied","Data":"fca3c03d125e6d153e0fbe53f31eaeba044dd1d7577420149c2c262392f570ff"}
Sep 19 06:07:11.386266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.386264    2581 scope.go:117] "RemoveContainer" containerID="b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4"
Sep 19 06:07:11.398457 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.398440    2581 scope.go:117] "RemoveContainer" containerID="b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4"
Sep 19 06:07:11.398752 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:11.398727    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4\": container with ID starting with b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4 not found: ID does not exist" containerID="b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4"
Sep 19 06:07:11.398854 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.398759    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4"} err="failed to get container status \"b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4\": rpc error: code = NotFound desc = could not find container \"b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4\": container with ID starting with b3e41fadfb540d9d0b4cb1ac7e12bca9ec366bcdf4de931fc874791d26a9ccb4 not found: ID does not exist"
Sep 19 06:07:11.409721 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.409699    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"]
Sep 19 06:07:11.415245 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.415223    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2b3fb3-75c7c747fd-shvhq"]
Sep 19 06:07:11.442937 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.442916    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:11.442937 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:11.442937    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hcfz\" (UniqueName: \"kubernetes.io/projected/69baeff6-8513-4b8f-9a61-9fa599e5f8b6-kube-api-access-5hcfz\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:12.477870 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:12.477832    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bac417e-9ba6-4ee9-9f38-ba741425695a" path="/var/lib/kubelet/pods/2bac417e-9ba6-4ee9-9f38-ba741425695a/volumes"
Sep 19 06:07:12.478272 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:12.478257    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69baeff6-8513-4b8f-9a61-9fa599e5f8b6" path="/var/lib/kubelet/pods/69baeff6-8513-4b8f-9a61-9fa599e5f8b6/volumes"
Sep 19 06:07:14.353950 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:14.353912    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:07:22.240003 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:22.239969    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"]
Sep 19 06:07:25.703079 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:25.703020    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j" podUID="11d48a1a-e73b-4675-bae1-96f9fcec9e95" containerName="main" containerID="cri-o://2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0" gracePeriod=45
Sep 19 06:07:25.706367 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:25.706336    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq" podUID="3a83c1e8-77a4-4898-a2e9-89f4b6998554" containerName="main" containerID="cri-o://c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c" gracePeriod=45
Sep 19 06:07:25.721253 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:25.721224    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz" podUID="1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" containerName="main" containerID="cri-o://01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b" gracePeriod=45
Sep 19 06:07:25.730395 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:25.730370    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8" podUID="5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" containerName="main" containerID="cri-o://d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e" gracePeriod=45
Sep 19 06:07:26.017700 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.017674    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:07:26.049419 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.049388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 06:07:26.066384 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.066168    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 06:07:26.070811 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.070742    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:07:26.180698 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180665    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-dshm\") pod \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") "
Sep 19 06:07:26.180698 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180703    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-model-cache\") pod \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") "
Sep 19 06:07:26.180918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-dshm\") pod \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") "
Sep 19 06:07:26.180918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180768    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tmp-dir\") pod \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") "
Sep 19 06:07:26.180918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-model-cache\") pod \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") "
Sep 19 06:07:26.180918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180835    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-model-cache\") pod \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") "
Sep 19 06:07:26.180918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180871    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tls-certs\") pod \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") "
Sep 19 06:07:26.180918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180896    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2g44\" (UniqueName: \"kubernetes.io/projected/11d48a1a-e73b-4675-bae1-96f9fcec9e95-kube-api-access-d2g44\") pod \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") "
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z57kg\" (UniqueName: \"kubernetes.io/projected/3a83c1e8-77a4-4898-a2e9-89f4b6998554-kube-api-access-z57kg\") pod \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") "
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180950    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tls-certs\") pod \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") "
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.180980    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tmp-dir\") pod \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") "
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181001    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-home\") pod \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") "
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181004    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" (UID: "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181021    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-model-cache" (OuterVolumeSpecName: "model-cache") pod "11d48a1a-e73b-4675-bae1-96f9fcec9e95" (UID: "11d48a1a-e73b-4675-bae1-96f9fcec9e95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11d48a1a-e73b-4675-bae1-96f9fcec9e95" (UID: "11d48a1a-e73b-4675-bae1-96f9fcec9e95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-dshm\") pod \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") "
Sep 19 06:07:26.181284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181241    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" (UID: "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.181773 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" (UID: "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.181773 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-home" (OuterVolumeSpecName: "home") pod "3a83c1e8-77a4-4898-a2e9-89f4b6998554" (UID: "3a83c1e8-77a4-4898-a2e9-89f4b6998554"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.181773 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-home\") pod \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") "
Sep 19 06:07:26.181947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181814    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-home\") pod \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") "
Sep 19 06:07:26.181947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181844    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tls-certs\") pod \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") "
Sep 19 06:07:26.181947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-model-cache\") pod \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") "
Sep 19 06:07:26.181947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181898    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-494d7\" (UniqueName: \"kubernetes.io/projected/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-kube-api-access-494d7\") pod \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") "
Sep 19 06:07:26.181947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdfq2\" (UniqueName: \"kubernetes.io/projected/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-kube-api-access-jdfq2\") pod \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") "
Sep 19 06:07:26.182238 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181962    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-dshm\") pod \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") "
Sep 19 06:07:26.182238 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.181987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tls-certs\") pod \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\" (UID: \"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7\") "
Sep 19 06:07:26.182238 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182011    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-home\") pod \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\" (UID: \"11d48a1a-e73b-4675-bae1-96f9fcec9e95\") "
Sep 19 06:07:26.182238 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tmp-dir\") pod \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\" (UID: \"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13\") "
Sep 19 06:07:26.182238 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tmp-dir\") pod \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\" (UID: \"3a83c1e8-77a4-4898-a2e9-89f4b6998554\") "
Sep 19 06:07:26.182827 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182493    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.182827 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182525    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.182827 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182538    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.182827 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182549    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.182827 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182561    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.182827 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.182572    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.183684 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.183113    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a83c1e8-77a4-4898-a2e9-89f4b6998554" (UID: "3a83c1e8-77a4-4898-a2e9-89f4b6998554"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.183782 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.183747    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a83c1e8-77a4-4898-a2e9-89f4b6998554-kube-api-access-z57kg" (OuterVolumeSpecName: "kube-api-access-z57kg") pod "3a83c1e8-77a4-4898-a2e9-89f4b6998554" (UID: "3a83c1e8-77a4-4898-a2e9-89f4b6998554"). InnerVolumeSpecName "kube-api-access-z57kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:26.183947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.183763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-dshm" (OuterVolumeSpecName: "dshm") pod "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" (UID: "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.183947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.183793    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-home" (OuterVolumeSpecName: "home") pod "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" (UID: "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.183947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.183763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d48a1a-e73b-4675-bae1-96f9fcec9e95-kube-api-access-d2g44" (OuterVolumeSpecName: "kube-api-access-d2g44") pod "11d48a1a-e73b-4675-bae1-96f9fcec9e95" (UID: "11d48a1a-e73b-4675-bae1-96f9fcec9e95"). InnerVolumeSpecName "kube-api-access-d2g44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:26.183947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.183844    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11d48a1a-e73b-4675-bae1-96f9fcec9e95" (UID: "11d48a1a-e73b-4675-bae1-96f9fcec9e95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:26.184215 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" (UID: "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.184215 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184081    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-home" (OuterVolumeSpecName: "home") pod "11d48a1a-e73b-4675-bae1-96f9fcec9e95" (UID: "11d48a1a-e73b-4675-bae1-96f9fcec9e95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.184393 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a83c1e8-77a4-4898-a2e9-89f4b6998554" (UID: "3a83c1e8-77a4-4898-a2e9-89f4b6998554"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.184457 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-home" (OuterVolumeSpecName: "home") pod "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" (UID: "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.184739 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184701    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-dshm" (OuterVolumeSpecName: "dshm") pod "11d48a1a-e73b-4675-bae1-96f9fcec9e95" (UID: "11d48a1a-e73b-4675-bae1-96f9fcec9e95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.184891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184832    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-dshm" (OuterVolumeSpecName: "dshm") pod "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" (UID: "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.185108 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.184860    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" (UID: "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:26.185945 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.185921    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-dshm" (OuterVolumeSpecName: "dshm") pod "3a83c1e8-77a4-4898-a2e9-89f4b6998554" (UID: "3a83c1e8-77a4-4898-a2e9-89f4b6998554"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:26.186353 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.186329    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-kube-api-access-494d7" (OuterVolumeSpecName: "kube-api-access-494d7") pod "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" (UID: "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7"). InnerVolumeSpecName "kube-api-access-494d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:26.186353 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.186341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" (UID: "5cd54113-b6c3-4ecf-8217-e5f211a9d5b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:26.186467 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.186424    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-kube-api-access-jdfq2" (OuterVolumeSpecName: "kube-api-access-jdfq2") pod "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" (UID: "1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13"). InnerVolumeSpecName "kube-api-access-jdfq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:26.186625 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.186612    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a83c1e8-77a4-4898-a2e9-89f4b6998554" (UID: "3a83c1e8-77a4-4898-a2e9-89f4b6998554"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:26.283020 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.282989    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-494d7\" (UniqueName: \"kubernetes.io/projected/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-kube-api-access-494d7\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283020 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283016    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdfq2\" (UniqueName: \"kubernetes.io/projected/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-kube-api-access-jdfq2\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283026    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283060    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283069    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283076    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283083    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283090    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283096    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283103    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d48a1a-e73b-4675-bae1-96f9fcec9e95-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283111    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2g44\" (UniqueName: \"kubernetes.io/projected/11d48a1a-e73b-4675-bae1-96f9fcec9e95-kube-api-access-d2g44\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283119    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z57kg\" (UniqueName: \"kubernetes.io/projected/3a83c1e8-77a4-4898-a2e9-89f4b6998554-kube-api-access-z57kg\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283127    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283135    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11d48a1a-e73b-4675-bae1-96f9fcec9e95-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283141    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283148    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283154    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a83c1e8-77a4-4898-a2e9-89f4b6998554-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.283214 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.283162    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a83c1e8-77a4-4898-a2e9-89f4b6998554-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:26.470399 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.470365    2581 generic.go:356] "Generic (PLEG): container finished" podID="5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" containerID="d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e" exitCode=0
Sep 19 06:07:26.470577 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.470411    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"
Sep 19 06:07:26.470577 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.470454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8" event={"ID":"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7","Type":"ContainerDied","Data":"d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e"}
Sep 19 06:07:26.470577 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.470531    2581 scope.go:117] "RemoveContainer" containerID="d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e"
Sep 19 06:07:26.472112 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.472085    2581 generic.go:356] "Generic (PLEG): container finished" podID="1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" containerID="01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b" exitCode=0
Sep 19 06:07:26.473752 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.473728    2581 generic.go:356] "Generic (PLEG): container finished" podID="11d48a1a-e73b-4675-bae1-96f9fcec9e95" containerID="2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0" exitCode=0
Sep 19 06:07:26.474256 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.474237    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"
Sep 19 06:07:26.474391 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.474357    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"
Sep 19 06:07:26.476361 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.476337    2581 generic.go:356] "Generic (PLEG): container finished" podID="3a83c1e8-77a4-4898-a2e9-89f4b6998554" containerID="c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c" exitCode=0
Sep 19 06:07:26.476457 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.476412    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"
Sep 19 06:07:26.479750 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8" event={"ID":"5cd54113-b6c3-4ecf-8217-e5f211a9d5b7","Type":"ContainerDied","Data":"34ebf313fc1dc2a3bb529ac7c7d972c16186029803c504da8bf23f24f34541e9"}
Sep 19 06:07:26.479863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz" event={"ID":"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13","Type":"ContainerDied","Data":"01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b"}
Sep 19 06:07:26.479863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz" event={"ID":"1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13","Type":"ContainerDied","Data":"bfd7d79271a877ac4955d5b97c53964a480ee8fe3d103e56860976ee3c08c888"}
Sep 19 06:07:26.479863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j" event={"ID":"11d48a1a-e73b-4675-bae1-96f9fcec9e95","Type":"ContainerDied","Data":"2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0"}
Sep 19 06:07:26.479863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j" event={"ID":"11d48a1a-e73b-4675-bae1-96f9fcec9e95","Type":"ContainerDied","Data":"b39041fbd2983fa1386f873cecbfcd4d1dc3004ef8b120f2de4f696333567219"}
Sep 19 06:07:26.479863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq" event={"ID":"3a83c1e8-77a4-4898-a2e9-89f4b6998554","Type":"ContainerDied","Data":"c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c"}
Sep 19 06:07:26.479863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.479837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq" event={"ID":"3a83c1e8-77a4-4898-a2e9-89f4b6998554","Type":"ContainerDied","Data":"4f2290c62a1da3900148d813ed24ab49ed2438c0b6798c2d7aa864715d6e2f0c"}
Sep 19 06:07:26.483525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.482797    2581 scope.go:117] "RemoveContainer" containerID="d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e"
Sep 19 06:07:26.483525 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:26.483108    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e\": container with ID starting with d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e not found: ID does not exist" containerID="d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e"
Sep 19 06:07:26.483525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.483138    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e"} err="failed to get container status \"d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e\": rpc error: code = NotFound desc = could not find container \"d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e\": container with ID starting with d8a56ed67686ad1aefeb7667e837edb8f5fdf829a1c2016c169274a1630bfb5e not found: ID does not exist"
Sep 19 06:07:26.483525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.483160    2581 scope.go:117] "RemoveContainer" containerID="01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b"
Sep 19 06:07:26.498228 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.498211    2581 scope.go:117] "RemoveContainer" containerID="01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b"
Sep 19 06:07:26.498589 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:26.498561    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b\": container with ID starting with 01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b not found: ID does not exist" containerID="01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b"
Sep 19 06:07:26.498714 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.498597    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b"} err="failed to get container status \"01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b\": rpc error: code = NotFound desc = could not find container \"01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b\": container with ID starting with 01f1e00c88f21e038e979b65e6f3aa2626b31057c1d1336dffa22461b0d8144b not found: ID does not exist"
Sep 19 06:07:26.498714 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.498620    2581 scope.go:117] "RemoveContainer" containerID="2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0"
Sep 19 06:07:26.507987 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.507958    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"]
Sep 19 06:07:26.512417 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.512396    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/simulator-w4-3182b5-kserve-7877966c8d-nh9n8"]
Sep 19 06:07:26.515713 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.515697    2581 scope.go:117] "RemoveContainer" containerID="2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0"
Sep 19 06:07:26.516011 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:26.515992    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0\": container with ID starting with 2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0 not found: ID does not exist" containerID="2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0"
Sep 19 06:07:26.516098 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.516018    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0"} err="failed to get container status \"2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0\": rpc error: code = NotFound desc = could not find container \"2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0\": container with ID starting with 2523c95ec04a8b9e2713f93b91060e922125cad02735ff4c6617744bb2cf00d0 not found: ID does not exist"
Sep 19 06:07:26.516098 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.516059    2581 scope.go:117] "RemoveContainer" containerID="c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c"
Sep 19 06:07:26.521598 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.521567    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"]
Sep 19 06:07:26.525228 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.525206    2581 scope.go:117] "RemoveContainer" containerID="c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c"
Sep 19 06:07:26.525526 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:26.525501    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c\": container with ID starting with c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c not found: ID does not exist" containerID="c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c"
Sep 19 06:07:26.525636 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.525533    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c"} err="failed to get container status \"c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c\": rpc error: code = NotFound desc = could not find container \"c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c\": container with ID starting with c3fcc5c7f67215fea284ac605e2258a2addfdc2657f09a73734323935f61894c not found: ID does not exist"
Sep 19 06:07:26.525907 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.525885    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-2-w4-3182b5-kserve-8687b5665f-nvb5j"]
Sep 19 06:07:26.534459 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.534411    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"]
Sep 19 06:07:26.538642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.538620    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/distinct-w4-3182b5-kserve-6b6cb6ddd-4l9wq"]
Sep 19 06:07:26.547861 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.547834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"]
Sep 19 06:07:26.550262 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:26.550242    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/premium-w4-3182b5-kserve-776fb96d85-7n5vz"]
Sep 19 06:07:28.478476 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.478443    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d48a1a-e73b-4675-bae1-96f9fcec9e95" path="/var/lib/kubelet/pods/11d48a1a-e73b-4675-bae1-96f9fcec9e95/volumes"
Sep 19 06:07:28.478924 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.478816    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" path="/var/lib/kubelet/pods/1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13/volumes"
Sep 19 06:07:28.479175 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.479161    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a83c1e8-77a4-4898-a2e9-89f4b6998554" path="/var/lib/kubelet/pods/3a83c1e8-77a4-4898-a2e9-89f4b6998554/volumes"
Sep 19 06:07:28.479505 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.479493    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" path="/var/lib/kubelet/pods/5cd54113-b6c3-4ecf-8217-e5f211a9d5b7/volumes"
Sep 19 06:07:28.923963 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.923929    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"]
Sep 19 06:07:28.924205 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.924166    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" podUID="7b919a27-9c80-45f9-b898-d70afcc92af8" containerName="maas-api" containerID="cri-o://9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf" gracePeriod=30
Sep 19 06:07:28.982510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.982477    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"]
Sep 19 06:07:28.982721 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:28.982698    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f" podUID="c350701a-3d27-4e67-a454-e5a422bf1897" containerName="payload-processing" containerID="cri-o://64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f" gracePeriod=30
Sep 19 06:07:29.254016 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.253989    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 06:07:29.281889 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.281854    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 06:07:29.411180 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411144    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b919a27-9c80-45f9-b898-d70afcc92af8-openshift-service-ca\") pod \"7b919a27-9c80-45f9-b898-d70afcc92af8\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") "
Sep 19 06:07:29.411370 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411195    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-metrics-certs\") pod \"7b919a27-9c80-45f9-b898-d70afcc92af8\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") "
Sep 19 06:07:29.411370 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411221    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx4t7\" (UniqueName: \"kubernetes.io/projected/7b919a27-9c80-45f9-b898-d70afcc92af8-kube-api-access-dx4t7\") pod \"7b919a27-9c80-45f9-b898-d70afcc92af8\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") "
Sep 19 06:07:29.411370 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411333    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpg9g\" (UniqueName: \"kubernetes.io/projected/c350701a-3d27-4e67-a454-e5a422bf1897-kube-api-access-mpg9g\") pod \"c350701a-3d27-4e67-a454-e5a422bf1897\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") "
Sep 19 06:07:29.411609 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c350701a-3d27-4e67-a454-e5a422bf1897-plugins-config-volume\") pod \"c350701a-3d27-4e67-a454-e5a422bf1897\" (UID: \"c350701a-3d27-4e67-a454-e5a422bf1897\") "
Sep 19 06:07:29.411609 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411487    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls\") pod \"7b919a27-9c80-45f9-b898-d70afcc92af8\" (UID: \"7b919a27-9c80-45f9-b898-d70afcc92af8\") "
Sep 19 06:07:29.411609 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b919a27-9c80-45f9-b898-d70afcc92af8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b919a27-9c80-45f9-b898-d70afcc92af8" (UID: "7b919a27-9c80-45f9-b898-d70afcc92af8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:29.411808 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411786    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c350701a-3d27-4e67-a454-e5a422bf1897-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c350701a-3d27-4e67-a454-e5a422bf1897" (UID: "c350701a-3d27-4e67-a454-e5a422bf1897"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:29.411890 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.411802    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b919a27-9c80-45f9-b898-d70afcc92af8-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.413891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.413857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b919a27-9c80-45f9-b898-d70afcc92af8" (UID: "7b919a27-9c80-45f9-b898-d70afcc92af8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:29.413991 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.413883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b919a27-9c80-45f9-b898-d70afcc92af8-kube-api-access-dx4t7" (OuterVolumeSpecName: "kube-api-access-dx4t7") pod "7b919a27-9c80-45f9-b898-d70afcc92af8" (UID: "7b919a27-9c80-45f9-b898-d70afcc92af8"). InnerVolumeSpecName "kube-api-access-dx4t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:29.414073 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.413998    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7b919a27-9c80-45f9-b898-d70afcc92af8" (UID: "7b919a27-9c80-45f9-b898-d70afcc92af8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:29.414163 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.414146    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c350701a-3d27-4e67-a454-e5a422bf1897-kube-api-access-mpg9g" (OuterVolumeSpecName: "kube-api-access-mpg9g") pod "c350701a-3d27-4e67-a454-e5a422bf1897" (UID: "c350701a-3d27-4e67-a454-e5a422bf1897"). InnerVolumeSpecName "kube-api-access-mpg9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:29.498992 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.498918    2581 generic.go:356] "Generic (PLEG): container finished" podID="c350701a-3d27-4e67-a454-e5a422bf1897" containerID="64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f" exitCode=0
Sep 19 06:07:29.498992 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.498983    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"
Sep 19 06:07:29.499501 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.498992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f" event={"ID":"c350701a-3d27-4e67-a454-e5a422bf1897","Type":"ContainerDied","Data":"64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f"}
Sep 19 06:07:29.499501 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.499075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f" event={"ID":"c350701a-3d27-4e67-a454-e5a422bf1897","Type":"ContainerDied","Data":"ab6c412b3a7860c5d0e51b8bcc482a7a8abc61ed7d91a74549745e8015a16a2e"}
Sep 19 06:07:29.499501 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.499103    2581 scope.go:117] "RemoveContainer" containerID="64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f"
Sep 19 06:07:29.500553 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.500528    2581 generic.go:356] "Generic (PLEG): container finished" podID="7b919a27-9c80-45f9-b898-d70afcc92af8" containerID="9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf" exitCode=0
Sep 19 06:07:29.500673 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.500571    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"
Sep 19 06:07:29.500673 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.500633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" event={"ID":"7b919a27-9c80-45f9-b898-d70afcc92af8","Type":"ContainerDied","Data":"9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf"}
Sep 19 06:07:29.500795 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.500687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr" event={"ID":"7b919a27-9c80-45f9-b898-d70afcc92af8","Type":"ContainerDied","Data":"c517f7bef216fbd23640b2ca301a958a07b153c43c558da71aaf4364922ec170"}
Sep 19 06:07:29.508829 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.508811    2581 scope.go:117] "RemoveContainer" containerID="64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f"
Sep 19 06:07:29.509146 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:29.509125    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f\": container with ID starting with 64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f not found: ID does not exist" containerID="64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f"
Sep 19 06:07:29.509236 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.509152    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f"} err="failed to get container status \"64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f\": rpc error: code = NotFound desc = could not find container \"64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f\": container with ID starting with 64ae33e29f62f0fe49a5f14097a55c026833589e75a869cac4e9dfc492b7e43f not found: ID does not exist"
Sep 19 06:07:29.509236 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.509169    2581 scope.go:117] "RemoveContainer" containerID="9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf"
Sep 19 06:07:29.512535 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.512519    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.512535 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.512537    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b919a27-9c80-45f9-b898-d70afcc92af8-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.512667 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.512546    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx4t7\" (UniqueName: \"kubernetes.io/projected/7b919a27-9c80-45f9-b898-d70afcc92af8-kube-api-access-dx4t7\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.512667 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.512554    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpg9g\" (UniqueName: \"kubernetes.io/projected/c350701a-3d27-4e67-a454-e5a422bf1897-kube-api-access-mpg9g\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.512667 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.512565    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c350701a-3d27-4e67-a454-e5a422bf1897-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.516949 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.516935    2581 scope.go:117] "RemoveContainer" containerID="9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf"
Sep 19 06:07:29.517231 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:29.517213    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf\": container with ID starting with 9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf not found: ID does not exist" containerID="9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf"
Sep 19 06:07:29.517281 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.517237    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf"} err="failed to get container status \"9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf\": rpc error: code = NotFound desc = could not find container \"9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf\": container with ID starting with 9b98c7def58d73e742a6cd04f60fe9a8caa7a45951e71d102f697d7f680318bf not found: ID does not exist"
Sep 19 06:07:29.523909 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.523888    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"]
Sep 19 06:07:29.526839 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.526820    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-836c16-5c6cdc5557-hjf6f"]
Sep 19 06:07:29.535867 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.535836    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"]
Sep 19 06:07:29.539968 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:29.539933    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-836c16-7fc9bc6744-p2nqr"]
Sep 19 06:07:30.476920 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:30.476884    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b919a27-9c80-45f9-b898-d70afcc92af8" path="/var/lib/kubelet/pods/7b919a27-9c80-45f9-b898-d70afcc92af8/volumes"
Sep 19 06:07:30.477291 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:30.477277    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c350701a-3d27-4e67-a454-e5a422bf1897" path="/var/lib/kubelet/pods/c350701a-3d27-4e67-a454-e5a422bf1897/volumes"
Sep 19 06:07:35.286375 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:35.286332    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"]
Sep 19 06:07:35.287165 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:35.287133    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" containerID="cri-o://daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801" gracePeriod=30
Sep 19 06:07:35.355272 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:35.355239    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:07:36.157876 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:36.157834    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 06:07:36.158066 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:36.157909    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 06:07:37.260226 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.260124    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh" podUID="309764c0-3a4a-45e5-adbe-63a5eec2cc74" containerName="main" containerID="cri-o://4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c" gracePeriod=45
Sep 19 06:07:37.514293 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.514236    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:07:37.549417 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.549383    2581 generic.go:356] "Generic (PLEG): container finished" podID="309764c0-3a4a-45e5-adbe-63a5eec2cc74" containerID="4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c" exitCode=0
Sep 19 06:07:37.549559 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.549424    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"
Sep 19 06:07:37.549559 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.549479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh" event={"ID":"309764c0-3a4a-45e5-adbe-63a5eec2cc74","Type":"ContainerDied","Data":"4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c"}
Sep 19 06:07:37.549559 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.549518    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh" event={"ID":"309764c0-3a4a-45e5-adbe-63a5eec2cc74","Type":"ContainerDied","Data":"350cfcf83dcdbb6e445631b7ad31fc4f7d404ca0d4719ebd5c812543b297c341"}
Sep 19 06:07:37.549559 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.549536    2581 scope.go:117] "RemoveContainer" containerID="4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c"
Sep 19 06:07:37.558273 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.558253    2581 scope.go:117] "RemoveContainer" containerID="4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c"
Sep 19 06:07:37.558553 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:37.558531    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c\": container with ID starting with 4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c not found: ID does not exist" containerID="4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c"
Sep 19 06:07:37.558625 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.558557    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c"} err="failed to get container status \"4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c\": rpc error: code = NotFound desc = could not find container \"4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c\": container with ID starting with 4271b6d0a585935b1757db20d821e282683b7ecdf3d24e3a619cf6d0a937169c not found: ID does not exist"
Sep 19 06:07:37.684095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684064    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-dshm\") pod \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") "
Sep 19 06:07:37.684233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tmp-dir\") pod \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") "
Sep 19 06:07:37.684233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684189    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tls-certs\") pod \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") "
Sep 19 06:07:37.684233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684213    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-home\") pod \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") "
Sep 19 06:07:37.684380 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684247    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-model-cache\") pod \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") "
Sep 19 06:07:37.684380 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684276    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85487\" (UniqueName: \"kubernetes.io/projected/309764c0-3a4a-45e5-adbe-63a5eec2cc74-kube-api-access-85487\") pod \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\" (UID: \"309764c0-3a4a-45e5-adbe-63a5eec2cc74\") "
Sep 19 06:07:37.684483 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684388    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "309764c0-3a4a-45e5-adbe-63a5eec2cc74" (UID: "309764c0-3a4a-45e5-adbe-63a5eec2cc74"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:37.684541 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-home" (OuterVolumeSpecName: "home") pod "309764c0-3a4a-45e5-adbe-63a5eec2cc74" (UID: "309764c0-3a4a-45e5-adbe-63a5eec2cc74"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:37.684594 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684535    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-model-cache" (OuterVolumeSpecName: "model-cache") pod "309764c0-3a4a-45e5-adbe-63a5eec2cc74" (UID: "309764c0-3a4a-45e5-adbe-63a5eec2cc74"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:37.684691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684673    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:37.684751 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684711    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:37.684751 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.684720    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:37.686405 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.686373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-dshm" (OuterVolumeSpecName: "dshm") pod "309764c0-3a4a-45e5-adbe-63a5eec2cc74" (UID: "309764c0-3a4a-45e5-adbe-63a5eec2cc74"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:37.686518 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.686434    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "309764c0-3a4a-45e5-adbe-63a5eec2cc74" (UID: "309764c0-3a4a-45e5-adbe-63a5eec2cc74"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:37.686518 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.686478    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309764c0-3a4a-45e5-adbe-63a5eec2cc74-kube-api-access-85487" (OuterVolumeSpecName: "kube-api-access-85487") pod "309764c0-3a4a-45e5-adbe-63a5eec2cc74" (UID: "309764c0-3a4a-45e5-adbe-63a5eec2cc74"). InnerVolumeSpecName "kube-api-access-85487". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:37.785149 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.785126    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/309764c0-3a4a-45e5-adbe-63a5eec2cc74-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:37.785271 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.785151    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/309764c0-3a4a-45e5-adbe-63a5eec2cc74-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:37.785271 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.785162    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85487\" (UniqueName: \"kubernetes.io/projected/309764c0-3a4a-45e5-adbe-63a5eec2cc74-kube-api-access-85487\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:37.870440 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.870411    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"]
Sep 19 06:07:37.872177 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:37.872157    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/auth-test-model-888f75-kserve-676c44595f-4d4rh"]
Sep 19 06:07:38.477902 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:38.477867    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="309764c0-3a4a-45e5-adbe-63a5eec2cc74" path="/var/lib/kubelet/pods/309764c0-3a4a-45e5-adbe-63a5eec2cc74/volumes"
Sep 19 06:07:40.543527 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.543504    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 06:07:40.577064 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.576951    2581 generic.go:356] "Generic (PLEG): container finished" podID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerID="daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801" exitCode=0
Sep 19 06:07:40.577064 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.577004    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"
Sep 19 06:07:40.577064 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.577050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" event={"ID":"607484dc-1c8a-4094-9b21-7741ea3c4f53","Type":"ContainerDied","Data":"daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801"}
Sep 19 06:07:40.577333 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.577146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql" event={"ID":"607484dc-1c8a-4094-9b21-7741ea3c4f53","Type":"ContainerDied","Data":"ed88f3f74345ef649c238e99621ceaf15413b472b0048538ab0b9844559dfbb4"}
Sep 19 06:07:40.577333 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.577179    2581 scope.go:117] "RemoveContainer" containerID="daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801"
Sep 19 06:07:40.585892 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.585874    2581 scope.go:117] "RemoveContainer" containerID="f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"
Sep 19 06:07:40.595455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.595433    2581 scope.go:117] "RemoveContainer" containerID="daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801"
Sep 19 06:07:40.595792 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:40.595755    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801\": container with ID starting with daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801 not found: ID does not exist" containerID="daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801"
Sep 19 06:07:40.595870 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.595806    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801"} err="failed to get container status \"daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801\": rpc error: code = NotFound desc = could not find container \"daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801\": container with ID starting with daab53a27429b847e6bfaeee426753f6b522c1057adb601edc5252886b5cc801 not found: ID does not exist"
Sep 19 06:07:40.595870 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.595829    2581 scope.go:117] "RemoveContainer" containerID="f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"
Sep 19 06:07:40.596198 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:07:40.596171    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498\": container with ID starting with f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498 not found: ID does not exist" containerID="f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"
Sep 19 06:07:40.596305 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.596204    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498"} err="failed to get container status \"f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498\": rpc error: code = NotFound desc = could not find container \"f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498\": container with ID starting with f75c470a9823b7800461907f7120a1d2610efca346ddeb96d2e6a7bf81d19498 not found: ID does not exist"
Sep 19 06:07:40.610626 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/607484dc-1c8a-4094-9b21-7741ea3c4f53-istiod-ca-cert\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610658    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-credential-socket\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610685    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-podinfo\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610770 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610703    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-data\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610770 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610743    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-token\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610770 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610757    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-socket\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610933 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610775    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-certs\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610933 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610822    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-envoy\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.610933 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610850    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqv5d\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-kube-api-access-tqv5d\") pod \"607484dc-1c8a-4094-9b21-7741ea3c4f53\" (UID: \"607484dc-1c8a-4094-9b21-7741ea3c4f53\") "
Sep 19 06:07:40.611111 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.610957    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/607484dc-1c8a-4094-9b21-7741ea3c4f53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:40.611111 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611000    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-data" (OuterVolumeSpecName: "istio-data") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:40.611111 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611009    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:40.611279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611137    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:40.611279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611178    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-credential-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.611279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611201    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-data\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.611279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611218    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/607484dc-1c8a-4094-9b21-7741ea3c4f53-istiod-ca-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.611279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.611273    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:40.613123 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.613097    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-kube-api-access-tqv5d" (OuterVolumeSpecName: "kube-api-access-tqv5d") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "kube-api-access-tqv5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:40.613233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.613160    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:07:40.613233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.613190    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-token" (OuterVolumeSpecName: "istio-token") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:40.613233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.613195    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "607484dc-1c8a-4094-9b21-7741ea3c4f53" (UID: "607484dc-1c8a-4094-9b21-7741ea3c4f53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:40.712419 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.712381    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-envoy\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.712419 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.712412    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqv5d\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-kube-api-access-tqv5d\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.712419 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.712423    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-podinfo\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.712657 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.712433    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/607484dc-1c8a-4094-9b21-7741ea3c4f53-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.712657 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.712443    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.712657 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.712451    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/607484dc-1c8a-4094-9b21-7741ea3c4f53-workload-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:07:40.898886 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.898857    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"]
Sep 19 06:07:40.904393 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:40.904365    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-836c16-openshift-default-758797f458-skhql"]
Sep 19 06:07:42.476381 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:42.476346    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" path="/var/lib/kubelet/pods/607484dc-1c8a-4094-9b21-7741ea3c4f53/volumes"
Sep 19 06:07:45.830210 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830180    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"]
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830502    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830514    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830521    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830527    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830535    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a83c1e8-77a4-4898-a2e9-89f4b6998554" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830541    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a83c1e8-77a4-4898-a2e9-89f4b6998554" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830550    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69baeff6-8513-4b8f-9a61-9fa599e5f8b6" containerName="payload-pre-processing"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830555    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69baeff6-8513-4b8f-9a61-9fa599e5f8b6" containerName="payload-pre-processing"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830565    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309764c0-3a4a-45e5-adbe-63a5eec2cc74" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830570    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="309764c0-3a4a-45e5-adbe-63a5eec2cc74" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830578    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b919a27-9c80-45f9-b898-d70afcc92af8" containerName="maas-api"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830583    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b919a27-9c80-45f9-b898-d70afcc92af8" containerName="maas-api"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830589    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d48a1a-e73b-4675-bae1-96f9fcec9e95" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830593    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d48a1a-e73b-4675-bae1-96f9fcec9e95" containerName="main"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830601    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c350701a-3d27-4e67-a454-e5a422bf1897" containerName="payload-processing"
Sep 19 06:07:45.830599 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830606    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c350701a-3d27-4e67-a454-e5a422bf1897" containerName="payload-processing"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830613    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" containerName="main"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830618    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" containerName="main"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830681    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea96e3c-a4dc-44b0-9d0d-ecae36f27e13" containerName="main"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830690    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b919a27-9c80-45f9-b898-d70afcc92af8" containerName="maas-api"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830699    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cd54113-b6c3-4ecf-8217-e5f211a9d5b7" containerName="main"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830707    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830715    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c350701a-3d27-4e67-a454-e5a422bf1897" containerName="payload-processing"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830723    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69baeff6-8513-4b8f-9a61-9fa599e5f8b6" containerName="payload-pre-processing"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830731    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a83c1e8-77a4-4898-a2e9-89f4b6998554" containerName="main"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830738    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d48a1a-e73b-4675-bae1-96f9fcec9e95" containerName="main"
Sep 19 06:07:45.831159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.830746    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="309764c0-3a4a-45e5-adbe-63a5eec2cc74" containerName="main"
Sep 19 06:07:45.833392 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.833378    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.838458 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.838435    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"openshift-service-ca.crt\""
Sep 19 06:07:45.839832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.839615    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"kube-root-ca.crt\""
Sep 19 06:07:45.839832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.839641    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"sub-test-model-888f75-kserve-self-signed-certs\""
Sep 19 06:07:45.839832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.839703    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"default-dockercfg-v9psp\""
Sep 19 06:07:45.845740 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.845717    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"]
Sep 19 06:07:45.854788 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.854745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tls-certs\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.854892 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.854820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97wlr\" (UniqueName: \"kubernetes.io/projected/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-kube-api-access-97wlr\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.854892 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.854880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-model-cache\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.855021 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.854978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-dshm\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.855113 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.855018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tmp-dir\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.855113 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.855060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-home\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.955858 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.955813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tls-certs\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.955858 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.955857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97wlr\" (UniqueName: \"kubernetes.io/projected/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-kube-api-access-97wlr\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956104 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.955893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-model-cache\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956104 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.955955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-dshm\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956104 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.955977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tmp-dir\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956104 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.956001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-home\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956429 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.956406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-home\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956566 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.956470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-model-cache\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.956566 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.956497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tmp-dir\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.958416 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.958400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-dshm\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.958574 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.958558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tls-certs\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:45.963415 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:45.963397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97wlr\" (UniqueName: \"kubernetes.io/projected/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-kube-api-access-97wlr\") pod \"sub-test-model-888f75-kserve-cc548496c-dmqxq\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") " pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:46.144026 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:46.143949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:46.414001 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:46.413977    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"]
Sep 19 06:07:46.415918 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:07:46.415877    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bcc1d6c_8e01_4bcd_8479_e16fd54e5109.slice/crio-284dad431debafb4fe0b7236478619612c4edbab36bef4d68af22cdd1074a0a9 WatchSource:0}: Error finding container 284dad431debafb4fe0b7236478619612c4edbab36bef4d68af22cdd1074a0a9: Status 404 returned error can't find the container with id 284dad431debafb4fe0b7236478619612c4edbab36bef4d68af22cdd1074a0a9
Sep 19 06:07:46.610962 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:46.610922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq" event={"ID":"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109","Type":"ContainerStarted","Data":"2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755"}
Sep 19 06:07:46.611154 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:46.610973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq" event={"ID":"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109","Type":"ContainerStarted","Data":"284dad431debafb4fe0b7236478619612c4edbab36bef4d68af22cdd1074a0a9"}
Sep 19 06:07:46.627823 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:46.627778    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq" podStartSLOduration=1.6277637839999999 podStartE2EDuration="1.627763784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:07:46.626134256 +0000 UTC m=+1612.236814246" watchObservedRunningTime="2026-09-19 06:07:46.627763784 +0000 UTC m=+1612.238443780"
Sep 19 06:07:50.848991 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:50.848945    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:07:56.144374 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:56.144337    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:56.144772 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:56.144390    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:56.157435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:56.157414    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:56.670679 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:56.670643    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:07:56.680657 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:07:56.680635    2581 scope.go:117] "RemoveContainer" containerID="b587fa4536802ed64c8b5fd056a7a1576363a7a12b85678a2b77e298e261d0b3"
Sep 19 06:08:38.345257 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.345220    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"]
Sep 19 06:08:38.345742 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.345559    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy"
Sep 19 06:08:38.345742 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.345570    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy"
Sep 19 06:08:38.345742 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.345624    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="607484dc-1c8a-4094-9b21-7741ea3c4f53" containerName="istio-proxy"
Sep 19 06:08:38.348769 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.348749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.352858 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.352832    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"sub-test-model-f1aad1-kserve-self-signed-certs\""
Sep 19 06:08:38.352977 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.352834    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"openshift-service-ca.crt\""
Sep 19 06:08:38.352977 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.352834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"default-dockercfg-kpxlv\""
Sep 19 06:08:38.352977 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.352873    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"kube-root-ca.crt\""
Sep 19 06:08:38.357084 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.357063    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"]
Sep 19 06:08:38.399619 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.399593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-tmp-dir\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.399758 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.399640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2tf9\" (UniqueName: \"kubernetes.io/projected/23c34501-49fb-457b-a2ce-594280700e57-kube-api-access-n2tf9\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.399758 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.399657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-dshm\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.399862 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.399750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-home\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.399862 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.399786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-model-cache\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.399862 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.399843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23c34501-49fb-457b-a2ce-594280700e57-tls-certs\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.500459 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.500423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2tf9\" (UniqueName: \"kubernetes.io/projected/23c34501-49fb-457b-a2ce-594280700e57-kube-api-access-n2tf9\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.500459 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.500459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-dshm\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.500696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.500487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-home\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.500696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.500534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-model-cache\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.500696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.500596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23c34501-49fb-457b-a2ce-594280700e57-tls-certs\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.500696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.500682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-tmp-dir\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.501130 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.501103    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-tmp-dir\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.501231 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.501184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-home\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.501310 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.501237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-model-cache\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.503536 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.503508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-dshm\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.503926 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.503907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23c34501-49fb-457b-a2ce-594280700e57-tls-certs\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.508459 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.508439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2tf9\" (UniqueName: \"kubernetes.io/projected/23c34501-49fb-457b-a2ce-594280700e57-kube-api-access-n2tf9\") pod \"sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.658476 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.658407    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:38.932062 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:38.932020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"]
Sep 19 06:08:38.933664 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:08:38.933638    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23c34501_49fb_457b_a2ce_594280700e57.slice/crio-bda5e46e1337860b93a3fe322afd64fbf4ae7b6a73cd99c02ec0c9d528f19732 WatchSource:0}: Error finding container bda5e46e1337860b93a3fe322afd64fbf4ae7b6a73cd99c02ec0c9d528f19732: Status 404 returned error can't find the container with id bda5e46e1337860b93a3fe322afd64fbf4ae7b6a73cd99c02ec0c9d528f19732
Sep 19 06:08:39.877262 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:39.877225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" event={"ID":"23c34501-49fb-457b-a2ce-594280700e57","Type":"ContainerStarted","Data":"43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce"}
Sep 19 06:08:39.877702 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:39.877272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" event={"ID":"23c34501-49fb-457b-a2ce-594280700e57","Type":"ContainerStarted","Data":"bda5e46e1337860b93a3fe322afd64fbf4ae7b6a73cd99c02ec0c9d528f19732"}
Sep 19 06:08:39.893998 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:39.893950    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" podStartSLOduration=1.893935267 podStartE2EDuration="1.893935267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:08:39.893347815 +0000 UTC m=+1665.504027779" watchObservedRunningTime="2026-09-19 06:08:39.893935267 +0000 UTC m=+1665.504615231"
Sep 19 06:08:42.241814 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:42.241777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:08:48.659376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:48.659347    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:48.659870 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:48.659409    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:48.672135 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:48.672114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:48.937964 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:48.937879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:08:58.049689 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:08:58.049651    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:09:13.494318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:13.494281    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:09:22.722279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:22.719114    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"]
Sep 19 06:09:23.722321 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:23.722288    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"]
Sep 19 06:09:25.138087 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:25.138052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:09:32.150828 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.150787    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"]
Sep 19 06:09:32.153856 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.153831    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.156309 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.156284    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-888f75\"/\"rate-test-model-888f75-kserve-self-signed-certs\""
Sep 19 06:09:32.163315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.163288    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"]
Sep 19 06:09:32.177625 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.177594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tls-certs\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.177764 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.177650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-dshm\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.177764 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.177709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tmp-dir\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.177907 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.177779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-kube-api-access-bn764\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.177907 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.177828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-home\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.177907 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.177859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-model-cache\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279188 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-home\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-model-cache\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tls-certs\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-dshm\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tmp-dir\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-kube-api-access-bn764\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279635 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-home\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279681 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-model-cache\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.279728 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.279685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tmp-dir\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.281682 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.281662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-dshm\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.282077 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.282058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tls-certs\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.287118 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.287095    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-kube-api-access-bn764\") pod \"rate-test-model-888f75-kserve-5784f884cf-ccbgj\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") " pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.464167 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.464085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:32.736976 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:32.736949    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"]
Sep 19 06:09:32.738770 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:09:32.738738    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2165f2f5_4da5_4c06_94ba_1b9a9d74edeb.slice/crio-d436a45bb5c9ac6bb8020583b8eb69d918792d78b3cce1c5e644a82e7a8e171f WatchSource:0}: Error finding container d436a45bb5c9ac6bb8020583b8eb69d918792d78b3cce1c5e644a82e7a8e171f: Status 404 returned error can't find the container with id d436a45bb5c9ac6bb8020583b8eb69d918792d78b3cce1c5e644a82e7a8e171f
Sep 19 06:09:33.141832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:33.141797    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj" event={"ID":"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb","Type":"ContainerStarted","Data":"8e4e7879a8f5f383e2d785596f39ff5638425fac14f0befad4e858670ffb2773"}
Sep 19 06:09:33.141832 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:33.141838    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj" event={"ID":"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb","Type":"ContainerStarted","Data":"d436a45bb5c9ac6bb8020583b8eb69d918792d78b3cce1c5e644a82e7a8e171f"}
Sep 19 06:09:33.158967 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:33.158917    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj" podStartSLOduration=1.158901644 podStartE2EDuration="1.158901644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:09:33.157288145 +0000 UTC m=+1718.767968135" watchObservedRunningTime="2026-09-19 06:09:33.158901644 +0000 UTC m=+1718.769581620"
Sep 19 06:09:34.841410 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:34.841369    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:09:37.747266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:37.747222    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq" podUID="4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" containerName="main" containerID="cri-o://2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755" gracePeriod=45
Sep 19 06:09:38.004909 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.004848    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:09:38.134356 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tls-certs\") pod \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") "
Sep 19 06:09:38.134356 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-dshm\") pod \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") "
Sep 19 06:09:38.134638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tmp-dir\") pod \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") "
Sep 19 06:09:38.134638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-home\") pod \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") "
Sep 19 06:09:38.134638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134604    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97wlr\" (UniqueName: \"kubernetes.io/projected/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-kube-api-access-97wlr\") pod \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") "
Sep 19 06:09:38.134808 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134652    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" (UID: "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:38.134808 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134695    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-model-cache\") pod \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\" (UID: \"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109\") "
Sep 19 06:09:38.134920 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134838    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-home" (OuterVolumeSpecName: "home") pod "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" (UID: "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:38.135018 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.134994    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-model-cache" (OuterVolumeSpecName: "model-cache") pod "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" (UID: "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:38.135302 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.135277    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:38.135302 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.135302    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:38.135443 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.135316    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:38.136751 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.136721    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-dshm" (OuterVolumeSpecName: "dshm") pod "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" (UID: "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:38.136831 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.136757    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" (UID: "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:09:38.137017 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.136993    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-kube-api-access-97wlr" (OuterVolumeSpecName: "kube-api-access-97wlr") pod "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" (UID: "4bcc1d6c-8e01-4bcd-8479-e16fd54e5109"). InnerVolumeSpecName "kube-api-access-97wlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:09:38.171749 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.171717    2581 generic.go:356] "Generic (PLEG): container finished" podID="4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" containerID="2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755" exitCode=0
Sep 19 06:09:38.171749 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.171749    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"
Sep 19 06:09:38.171926 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.171771    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq" event={"ID":"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109","Type":"ContainerDied","Data":"2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755"}
Sep 19 06:09:38.171926 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.171806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq" event={"ID":"4bcc1d6c-8e01-4bcd-8479-e16fd54e5109","Type":"ContainerDied","Data":"284dad431debafb4fe0b7236478619612c4edbab36bef4d68af22cdd1074a0a9"}
Sep 19 06:09:38.171926 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.171823    2581 scope.go:117] "RemoveContainer" containerID="2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755"
Sep 19 06:09:38.181276 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.181256    2581 scope.go:117] "RemoveContainer" containerID="2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755"
Sep 19 06:09:38.181554 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:09:38.181532    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755\": container with ID starting with 2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755 not found: ID does not exist" containerID="2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755"
Sep 19 06:09:38.181616 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.181567    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755"} err="failed to get container status \"2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755\": rpc error: code = NotFound desc = could not find container \"2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755\": container with ID starting with 2d843cdb32ba64f4135134f2b8053e9e07bc78bf05e15e8f5b30763b8430d755 not found: ID does not exist"
Sep 19 06:09:38.195046 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.195005    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"]
Sep 19 06:09:38.198300 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.198280    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/sub-test-model-888f75-kserve-cc548496c-dmqxq"]
Sep 19 06:09:38.236648 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.236629    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97wlr\" (UniqueName: \"kubernetes.io/projected/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-kube-api-access-97wlr\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:38.236648 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.236648    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:38.236797 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.236660    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:38.477634 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.477600    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" path="/var/lib/kubelet/pods/4bcc1d6c-8e01-4bcd-8479-e16fd54e5109/volumes"
Sep 19 06:09:38.742490 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:38.742391    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" podUID="23c34501-49fb-457b-a2ce-594280700e57" containerName="main" containerID="cri-o://43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce" gracePeriod=45
Sep 19 06:09:39.003658 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.003599    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:09:39.142964 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.142934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23c34501-49fb-457b-a2ce-594280700e57-tls-certs\") pod \"23c34501-49fb-457b-a2ce-594280700e57\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") "
Sep 19 06:09:39.143190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.142990    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-home\") pod \"23c34501-49fb-457b-a2ce-594280700e57\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") "
Sep 19 06:09:39.143190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-model-cache\") pod \"23c34501-49fb-457b-a2ce-594280700e57\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") "
Sep 19 06:09:39.143190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143095    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2tf9\" (UniqueName: \"kubernetes.io/projected/23c34501-49fb-457b-a2ce-594280700e57-kube-api-access-n2tf9\") pod \"23c34501-49fb-457b-a2ce-594280700e57\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") "
Sep 19 06:09:39.143190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143123    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-tmp-dir\") pod \"23c34501-49fb-457b-a2ce-594280700e57\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") "
Sep 19 06:09:39.143190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143148    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-dshm\") pod \"23c34501-49fb-457b-a2ce-594280700e57\" (UID: \"23c34501-49fb-457b-a2ce-594280700e57\") "
Sep 19 06:09:39.143421 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-home" (OuterVolumeSpecName: "home") pod "23c34501-49fb-457b-a2ce-594280700e57" (UID: "23c34501-49fb-457b-a2ce-594280700e57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:39.143421 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-model-cache" (OuterVolumeSpecName: "model-cache") pod "23c34501-49fb-457b-a2ce-594280700e57" (UID: "23c34501-49fb-457b-a2ce-594280700e57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:39.143421 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143395    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23c34501-49fb-457b-a2ce-594280700e57" (UID: "23c34501-49fb-457b-a2ce-594280700e57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:39.143549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143528    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:39.143591 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143554    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:39.143591 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.143571    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:39.145376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.145348    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-dshm" (OuterVolumeSpecName: "dshm") pod "23c34501-49fb-457b-a2ce-594280700e57" (UID: "23c34501-49fb-457b-a2ce-594280700e57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:09:39.145473 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.145415    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23c34501-49fb-457b-a2ce-594280700e57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23c34501-49fb-457b-a2ce-594280700e57" (UID: "23c34501-49fb-457b-a2ce-594280700e57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:09:39.145473 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.145428    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c34501-49fb-457b-a2ce-594280700e57-kube-api-access-n2tf9" (OuterVolumeSpecName: "kube-api-access-n2tf9") pod "23c34501-49fb-457b-a2ce-594280700e57" (UID: "23c34501-49fb-457b-a2ce-594280700e57"). InnerVolumeSpecName "kube-api-access-n2tf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:09:39.178770 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.178739    2581 generic.go:356] "Generic (PLEG): container finished" podID="23c34501-49fb-457b-a2ce-594280700e57" containerID="43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce" exitCode=0
Sep 19 06:09:39.178905 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.178785    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"
Sep 19 06:09:39.178905 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.178809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" event={"ID":"23c34501-49fb-457b-a2ce-594280700e57","Type":"ContainerDied","Data":"43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce"}
Sep 19 06:09:39.178905 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.178845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" event={"ID":"23c34501-49fb-457b-a2ce-594280700e57","Type":"ContainerDied","Data":"bda5e46e1337860b93a3fe322afd64fbf4ae7b6a73cd99c02ec0c9d528f19732"}
Sep 19 06:09:39.178905 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.178866    2581 scope.go:117] "RemoveContainer" containerID="43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce"
Sep 19 06:09:39.188466 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.188446    2581 scope.go:117] "RemoveContainer" containerID="43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce"
Sep 19 06:09:39.188780 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:09:39.188760    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce\": container with ID starting with 43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce not found: ID does not exist" containerID="43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce"
Sep 19 06:09:39.188867 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.188791    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce"} err="failed to get container status \"43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce\": rpc error: code = NotFound desc = could not find container \"43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce\": container with ID starting with 43ec555be40c62b87263121089c86957810ddf14a5098ddeee2f6c48641147ce not found: ID does not exist"
Sep 19 06:09:39.199073 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.199051    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"]
Sep 19 06:09:39.202642 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.202619    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6"]
Sep 19 06:09:39.244202 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.244173    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23c34501-49fb-457b-a2ce-594280700e57-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:39.244202 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.244197    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2tf9\" (UniqueName: \"kubernetes.io/projected/23c34501-49fb-457b-a2ce-594280700e57-kube-api-access-n2tf9\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:39.244202 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:39.244206    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23c34501-49fb-457b-a2ce-594280700e57-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:09:40.477015 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:40.476979    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23c34501-49fb-457b-a2ce-594280700e57" path="/var/lib/kubelet/pods/23c34501-49fb-457b-a2ce-594280700e57/volumes"
Sep 19 06:09:42.464978 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:42.464941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:42.465410 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:42.464995    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:42.477690 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:42.477660    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:43.212662 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:43.212635    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:09:43.928213 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:09:43.928172    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-f1aad1/sub-test-model-f1aad1-kserve-bd5c7bc9f-mv6z6" podUID="23c34501-49fb-457b-a2ce-594280700e57" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Sep 19 06:10:21.032459 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032416    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"]
Sep 19 06:10:21.032891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032773    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c34501-49fb-457b-a2ce-594280700e57" containerName="main"
Sep 19 06:10:21.032891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032785    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c34501-49fb-457b-a2ce-594280700e57" containerName="main"
Sep 19 06:10:21.032891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032795    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" containerName="main"
Sep 19 06:10:21.032891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032801    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" containerName="main"
Sep 19 06:10:21.032891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032857    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bcc1d6c-8e01-4bcd-8479-e16fd54e5109" containerName="main"
Sep 19 06:10:21.032891 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.032864    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="23c34501-49fb-457b-a2ce-594280700e57" containerName="main"
Sep 19 06:10:21.035767 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.035751    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.039348 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.039325    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"kube-root-ca.crt\""
Sep 19 06:10:21.039484 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.039325    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"rate-test-model-f1aad1-kserve-self-signed-certs\""
Sep 19 06:10:21.039669 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.039652    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"default-dockercfg-kpxlv\""
Sep 19 06:10:21.039770 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.039683    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"openshift-service-ca.crt\""
Sep 19 06:10:21.043872 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.043485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"]
Sep 19 06:10:21.100584 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.100550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-model-cache\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.100744 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.100597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-dshm\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.100744 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.100622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-tmp-dir\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.100744 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.100665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/bb8603b9-c097-4614-921a-c334c15a813b-kube-api-access-wq4nt\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.100744 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.100692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-home\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.100887 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.100749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb8603b9-c097-4614-921a-c334c15a813b-tls-certs\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202189 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-dshm\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202189 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-tmp-dir\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202416 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/bb8603b9-c097-4614-921a-c334c15a813b-kube-api-access-wq4nt\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202416 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-home\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202529 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb8603b9-c097-4614-921a-c334c15a813b-tls-certs\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202529 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-model-cache\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202625 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-tmp-dir\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202812 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202783    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-home\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.202930 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.202799    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-model-cache\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.204723 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.204698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-dshm\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.205126 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.205111    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb8603b9-c097-4614-921a-c334c15a813b-tls-certs\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.210147 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.210122    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/bb8603b9-c097-4614-921a-c334c15a813b-kube-api-access-wq4nt\") pod \"rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.346879 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.346786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:21.618940 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:21.618912    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"]
Sep 19 06:10:21.620845 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:10:21.620817    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb8603b9_c097_4614_921a_c334c15a813b.slice/crio-3f4e05b522e707b5e1c3990951573340bf9ec010f208a94da2d1a8754c4a68af WatchSource:0}: Error finding container 3f4e05b522e707b5e1c3990951573340bf9ec010f208a94da2d1a8754c4a68af: Status 404 returned error can't find the container with id 3f4e05b522e707b5e1c3990951573340bf9ec010f208a94da2d1a8754c4a68af
Sep 19 06:10:22.390762 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:22.390727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9" event={"ID":"bb8603b9-c097-4614-921a-c334c15a813b","Type":"ContainerStarted","Data":"75a48b4d1ee9c88e105e06b2c3c4e0f054e484a159fe3765b538f234ced2374c"}
Sep 19 06:10:22.390762 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:22.390772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9" event={"ID":"bb8603b9-c097-4614-921a-c334c15a813b","Type":"ContainerStarted","Data":"3f4e05b522e707b5e1c3990951573340bf9ec010f208a94da2d1a8754c4a68af"}
Sep 19 06:10:22.408883 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:22.408833    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9" podStartSLOduration=1.4088150210000001 podStartE2EDuration="1.408815021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:10:22.407254545 +0000 UTC m=+1768.017934541" watchObservedRunningTime="2026-09-19 06:10:22.408815021 +0000 UTC m=+1768.019494986"
Sep 19 06:10:25.042719 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:25.042679    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:10:31.347018 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:31.346985    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:31.347481 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:31.347068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:31.359955 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:31.359932    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:31.449857 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:31.449832    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:10:47.355871 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:47.355830    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:10:56.533580 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:56.533546    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:10:56.538296 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:56.538271    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:10:56.540579 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:56.540563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:10:56.545336 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:56.545319    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:10:59.245503 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:10:59.245467    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:11:05.451338 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:05.451296    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"]
Sep 19 06:11:06.449666 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:06.449631    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"]
Sep 19 06:11:09.945874 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:09.945836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:11:12.723715 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.723665    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"]
Sep 19 06:11:12.728321 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.728300    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:12.731002 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.730981    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7142e3-dockercfg-2bn8b\""
Sep 19 06:11:12.731152 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.731109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7142e3\""
Sep 19 06:11:12.738116 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.738092    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"]
Sep 19 06:11:12.760765 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.760730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b18f557-0f00-4d84-a6d7-a7849f273e78-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:12.760915 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.760817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqsqt\" (UniqueName: \"kubernetes.io/projected/1b18f557-0f00-4d84-a6d7-a7849f273e78-kube-api-access-xqsqt\") pod \"payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:12.861413 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.861379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b18f557-0f00-4d84-a6d7-a7849f273e78-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:12.861568 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.861449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqsqt\" (UniqueName: \"kubernetes.io/projected/1b18f557-0f00-4d84-a6d7-a7849f273e78-kube-api-access-xqsqt\") pod \"payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:12.862087 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.862067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b18f557-0f00-4d84-a6d7-a7849f273e78-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:12.869371 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:12.869342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqsqt\" (UniqueName: \"kubernetes.io/projected/1b18f557-0f00-4d84-a6d7-a7849f273e78-kube-api-access-xqsqt\") pod \"payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:13.040111 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:13.040084    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:13.310095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:13.310061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"]
Sep 19 06:11:13.312420 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:11:13.312391    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b18f557_0f00_4d84_a6d7_a7849f273e78.slice/crio-5ec466b0ce165f3b81f576606471588e12a163b8943cf1c4fd6eb12541d8d2e6 WatchSource:0}: Error finding container 5ec466b0ce165f3b81f576606471588e12a163b8943cf1c4fd6eb12541d8d2e6: Status 404 returned error can't find the container with id 5ec466b0ce165f3b81f576606471588e12a163b8943cf1c4fd6eb12541d8d2e6
Sep 19 06:11:13.638099 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:13.637997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z" event={"ID":"1b18f557-0f00-4d84-a6d7-a7849f273e78","Type":"ContainerStarted","Data":"402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e"}
Sep 19 06:11:13.638099 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:13.638071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z" event={"ID":"1b18f557-0f00-4d84-a6d7-a7849f273e78","Type":"ContainerStarted","Data":"5ec466b0ce165f3b81f576606471588e12a163b8943cf1c4fd6eb12541d8d2e6"}
Sep 19 06:11:13.638318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:13.638109    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:13.652889 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:13.652839    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z" podStartSLOduration=1.652823376 podStartE2EDuration="1.652823376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:13.652164537 +0000 UTC m=+1819.262844542" watchObservedRunningTime="2026-09-19 06:11:13.652823376 +0000 UTC m=+1819.263503341"
Sep 19 06:11:20.471510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.471466    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj" podUID="2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" containerName="main" containerID="cri-o://8e4e7879a8f5f383e2d785596f39ff5638425fac14f0befad4e858670ffb2773" gracePeriod=45
Sep 19 06:11:20.672298 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.672246    2581 generic.go:356] "Generic (PLEG): container finished" podID="2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" containerID="8e4e7879a8f5f383e2d785596f39ff5638425fac14f0befad4e858670ffb2773" exitCode=0
Sep 19 06:11:20.672476 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.672387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj" event={"ID":"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb","Type":"ContainerDied","Data":"8e4e7879a8f5f383e2d785596f39ff5638425fac14f0befad4e858670ffb2773"}
Sep 19 06:11:20.730525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.730467    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:11:20.828863 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.828827    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-home\") pod \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") "
Sep 19 06:11:20.829069 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.828883    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tls-certs\") pod \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") "
Sep 19 06:11:20.829069 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.828934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-kube-api-access-bn764\") pod \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") "
Sep 19 06:11:20.829069 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.828951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-dshm\") pod \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") "
Sep 19 06:11:20.829069 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.828970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-model-cache\") pod \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") "
Sep 19 06:11:20.829069 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.829013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tmp-dir\") pod \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\" (UID: \"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb\") "
Sep 19 06:11:20.829372 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.829145    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-home" (OuterVolumeSpecName: "home") pod "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" (UID: "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:20.829372 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.829262    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" (UID: "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:20.829372 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.829349    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" (UID: "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:20.831271 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.831245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-dshm" (OuterVolumeSpecName: "dshm") pod "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" (UID: "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:20.831271 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.831252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" (UID: "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:11:20.831409 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.831328    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-kube-api-access-bn764" (OuterVolumeSpecName: "kube-api-access-bn764") pod "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" (UID: "2165f2f5-4da5-4c06-94ba-1b9a9d74edeb"). InnerVolumeSpecName "kube-api-access-bn764". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:11:20.929874 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.929848    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn764\" (UniqueName: \"kubernetes.io/projected/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-kube-api-access-bn764\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:20.929874 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.929869    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:20.930026 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.929879    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:20.930026 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.929888    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:20.930026 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.929895    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:20.930026 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:20.929903    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:21.448149 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.448105    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:11:21.469323 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.469288    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9" podUID="bb8603b9-c097-4614-921a-c334c15a813b" containerName="main" containerID="cri-o://75a48b4d1ee9c88e105e06b2c3c4e0f054e484a159fe3765b538f234ced2374c" gracePeriod=45
Sep 19 06:11:21.679085 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.679052    2581 generic.go:356] "Generic (PLEG): container finished" podID="bb8603b9-c097-4614-921a-c334c15a813b" containerID="75a48b4d1ee9c88e105e06b2c3c4e0f054e484a159fe3765b538f234ced2374c" exitCode=0
Sep 19 06:11:21.679484 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.679131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9" event={"ID":"bb8603b9-c097-4614-921a-c334c15a813b","Type":"ContainerDied","Data":"75a48b4d1ee9c88e105e06b2c3c4e0f054e484a159fe3765b538f234ced2374c"}
Sep 19 06:11:21.680485 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.680464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj" event={"ID":"2165f2f5-4da5-4c06-94ba-1b9a9d74edeb","Type":"ContainerDied","Data":"d436a45bb5c9ac6bb8020583b8eb69d918792d78b3cce1c5e644a82e7a8e171f"}
Sep 19 06:11:21.680588 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.680502    2581 scope.go:117] "RemoveContainer" containerID="8e4e7879a8f5f383e2d785596f39ff5638425fac14f0befad4e858670ffb2773"
Sep 19 06:11:21.680588 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.680473    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"
Sep 19 06:11:21.702649 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.702542    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"]
Sep 19 06:11:21.704227 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.704203    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-888f75/rate-test-model-888f75-kserve-5784f884cf-ccbgj"]
Sep 19 06:11:21.719542 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.719523    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:11:21.837341 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837313    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-model-cache\") pod \"bb8603b9-c097-4614-921a-c334c15a813b\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") "
Sep 19 06:11:21.837510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837356    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/bb8603b9-c097-4614-921a-c334c15a813b-kube-api-access-wq4nt\") pod \"bb8603b9-c097-4614-921a-c334c15a813b\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") "
Sep 19 06:11:21.837510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837392    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-home\") pod \"bb8603b9-c097-4614-921a-c334c15a813b\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") "
Sep 19 06:11:21.837510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837419    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-dshm\") pod \"bb8603b9-c097-4614-921a-c334c15a813b\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") "
Sep 19 06:11:21.837510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837466    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-tmp-dir\") pod \"bb8603b9-c097-4614-921a-c334c15a813b\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") "
Sep 19 06:11:21.837510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837500    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb8603b9-c097-4614-921a-c334c15a813b-tls-certs\") pod \"bb8603b9-c097-4614-921a-c334c15a813b\" (UID: \"bb8603b9-c097-4614-921a-c334c15a813b\") "
Sep 19 06:11:21.837807 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837637    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-home" (OuterVolumeSpecName: "home") pod "bb8603b9-c097-4614-921a-c334c15a813b" (UID: "bb8603b9-c097-4614-921a-c334c15a813b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:21.837807 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837652    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb8603b9-c097-4614-921a-c334c15a813b" (UID: "bb8603b9-c097-4614-921a-c334c15a813b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:21.837807 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837723    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb8603b9-c097-4614-921a-c334c15a813b" (UID: "bb8603b9-c097-4614-921a-c334c15a813b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:21.837948 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837851    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:21.837948 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837870    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:21.837948 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.837884    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:21.839626 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.839604    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8603b9-c097-4614-921a-c334c15a813b-kube-api-access-wq4nt" (OuterVolumeSpecName: "kube-api-access-wq4nt") pod "bb8603b9-c097-4614-921a-c334c15a813b" (UID: "bb8603b9-c097-4614-921a-c334c15a813b"). InnerVolumeSpecName "kube-api-access-wq4nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:11:21.839746 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.839704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-dshm" (OuterVolumeSpecName: "dshm") pod "bb8603b9-c097-4614-921a-c334c15a813b" (UID: "bb8603b9-c097-4614-921a-c334c15a813b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:11:21.839807 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.839750    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb8603b9-c097-4614-921a-c334c15a813b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb8603b9-c097-4614-921a-c334c15a813b" (UID: "bb8603b9-c097-4614-921a-c334c15a813b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:11:21.938923 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.938896    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/bb8603b9-c097-4614-921a-c334c15a813b-kube-api-access-wq4nt\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:21.938923 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.938921    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb8603b9-c097-4614-921a-c334c15a813b-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:21.939126 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:21.938931    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb8603b9-c097-4614-921a-c334c15a813b-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:11:22.477592 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:22.477557    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" path="/var/lib/kubelet/pods/2165f2f5-4da5-4c06-94ba-1b9a9d74edeb/volumes"
Sep 19 06:11:22.687709 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:22.687677    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9" event={"ID":"bb8603b9-c097-4614-921a-c334c15a813b","Type":"ContainerDied","Data":"3f4e05b522e707b5e1c3990951573340bf9ec010f208a94da2d1a8754c4a68af"}
Sep 19 06:11:22.687709 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:22.687696    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"
Sep 19 06:11:22.688251 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:22.687722    2581 scope.go:117] "RemoveContainer" containerID="75a48b4d1ee9c88e105e06b2c3c4e0f054e484a159fe3765b538f234ced2374c"
Sep 19 06:11:22.703376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:22.703353    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"]
Sep 19 06:11:22.713201 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:22.713176    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/rate-test-model-f1aad1-kserve-7bcb84f7d7-cg7z9"]
Sep 19 06:11:24.477615 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:24.477578    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8603b9-c097-4614-921a-c334c15a813b" path="/var/lib/kubelet/pods/bb8603b9-c097-4614-921a-c334c15a813b/volumes"
Sep 19 06:11:24.644211 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:24.644178    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:11:36.574487 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574450    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"]
Sep 19 06:11:36.574951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574797    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" containerName="main"
Sep 19 06:11:36.574951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574808    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" containerName="main"
Sep 19 06:11:36.574951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574821    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8603b9-c097-4614-921a-c334c15a813b" containerName="main"
Sep 19 06:11:36.574951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574826    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8603b9-c097-4614-921a-c334c15a813b" containerName="main"
Sep 19 06:11:36.574951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574881    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2165f2f5-4da5-4c06-94ba-1b9a9d74edeb" containerName="main"
Sep 19 06:11:36.574951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.574889    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8603b9-c097-4614-921a-c334c15a813b" containerName="main"
Sep 19 06:11:36.586351 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.586323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.589444 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.589408    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 06:11:36.589689 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.589667    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-ebb0f4-openshift-default-dockercfg-tqmjq\""
Sep 19 06:11:36.593056 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.593014    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"]
Sep 19 06:11:36.658760 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istiod-ca-cert\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.658760 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7wj2\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-kube-api-access-k7wj2\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659004 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-token\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659004 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-credential-socket\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659004 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-podinfo\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659004 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-envoy\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659004 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.658973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-certs\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659229 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.659015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-data\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.659229 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.659109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-socket\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760104 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istiod-ca-cert\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wj2\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-kube-api-access-k7wj2\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-token\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-credential-socket\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-podinfo\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-envoy\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-certs\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-data\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-socket\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-credential-socket\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-certs\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-data\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-socket\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.760953 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.760850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istiod-ca-cert\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.762816 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.762792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-podinfo\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.762918 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.762881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-envoy\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.770326 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.770300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7wj2\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-kube-api-access-k7wj2\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.774315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.774297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-token\") pod \"e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") " pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:36.904060 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:36.903964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:37.180454 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.180376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"]
Sep 19 06:11:37.184025 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:11:37.183997    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cde2ed_f3e9_4071_a87e_3bfdcaeec3fd.slice/crio-2b21d86002d2027db151fa65b6de50c3812e8722e8fe905dadc88ba973e3ff76 WatchSource:0}: Error finding container 2b21d86002d2027db151fa65b6de50c3812e8722e8fe905dadc88ba973e3ff76: Status 404 returned error can't find the container with id 2b21d86002d2027db151fa65b6de50c3812e8722e8fe905dadc88ba973e3ff76
Sep 19 06:11:37.186168 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.186135    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:11:37.186256 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.186201    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:11:37.186256 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.186230    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:11:37.764439 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.764395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" event={"ID":"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd","Type":"ContainerStarted","Data":"0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"}
Sep 19 06:11:37.764875 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.764461    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" event={"ID":"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd","Type":"ContainerStarted","Data":"2b21d86002d2027db151fa65b6de50c3812e8722e8fe905dadc88ba973e3ff76"}
Sep 19 06:11:37.785264 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.785217    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podStartSLOduration=1.78520218 podStartE2EDuration="1.78520218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:37.782939108 +0000 UTC m=+1843.393619089" watchObservedRunningTime="2026-09-19 06:11:37.78520218 +0000 UTC m=+1843.395882144"
Sep 19 06:11:37.904122 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:37.904090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:38.910089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:38.910058    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:39.774643 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:39.774607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:39.775527 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:39.775510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:42.316105 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.316065    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"]
Sep 19 06:11:42.320429 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.320395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.322828 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.322811    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-ebb0f4-serving-cert\""
Sep 19 06:11:42.331606 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.331582    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"]
Sep 19 06:11:42.408133 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.408101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-metrics-certs\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.408133 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.408138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00a89c41-8388-44c5-8b9d-00bfbea03f70-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.408343 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.408220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4qg\" (UniqueName: \"kubernetes.io/projected/00a89c41-8388-44c5-8b9d-00bfbea03f70-kube-api-access-gf4qg\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.408343 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.408242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.509082 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.509008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00a89c41-8388-44c5-8b9d-00bfbea03f70-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.509274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.509148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4qg\" (UniqueName: \"kubernetes.io/projected/00a89c41-8388-44c5-8b9d-00bfbea03f70-kube-api-access-gf4qg\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.509274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.509173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.509274 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.509203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-metrics-certs\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.509459 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:11:42.509292    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-serving-cert: secret "maas-api-e2e-ipp-b-ebb0f4-serving-cert" not found
Sep 19 06:11:42.509459 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:11:42.509355    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls podName:00a89c41-8388-44c5-8b9d-00bfbea03f70 nodeName:}" failed. No retries permitted until 2026-09-19 06:11:43.00933698 +0000 UTC m=+1848.620016925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls") pod "maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" (UID: "00a89c41-8388-44c5-8b9d-00bfbea03f70") : secret "maas-api-e2e-ipp-b-ebb0f4-serving-cert" not found
Sep 19 06:11:42.509776 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.509751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00a89c41-8388-44c5-8b9d-00bfbea03f70-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.511811 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.511788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-metrics-certs\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:42.517423 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:42.517402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4qg\" (UniqueName: \"kubernetes.io/projected/00a89c41-8388-44c5-8b9d-00bfbea03f70-kube-api-access-gf4qg\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:43.014304 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:43.014267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:43.017234 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:43.017209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:43.233283 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:43.233249    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:43.503966 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:43.503943    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"]
Sep 19 06:11:43.506105 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:11:43.506076    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a89c41_8388_44c5_8b9d_00bfbea03f70.slice/crio-4288a6fe682dada049468c82692e76059d5e61c626714a330da0d4f8da7d595d WatchSource:0}: Error finding container 4288a6fe682dada049468c82692e76059d5e61c626714a330da0d4f8da7d595d: Status 404 returned error can't find the container with id 4288a6fe682dada049468c82692e76059d5e61c626714a330da0d4f8da7d595d
Sep 19 06:11:43.507594 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:43.507577    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 06:11:43.800003 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:43.799971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" event={"ID":"00a89c41-8388-44c5-8b9d-00bfbea03f70","Type":"ContainerStarted","Data":"4288a6fe682dada049468c82692e76059d5e61c626714a330da0d4f8da7d595d"}
Sep 19 06:11:44.810293 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:44.810258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" event={"ID":"00a89c41-8388-44c5-8b9d-00bfbea03f70","Type":"ContainerStarted","Data":"c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21"}
Sep 19 06:11:44.810812 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:44.810337    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:44.826542 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:44.826499    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" podStartSLOduration=2.517643833 podStartE2EDuration="2.82648692s" totalImagesPullingTime="308.843087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:44.825808111 +0000 UTC m=+1850.436488108" watchObservedRunningTime="2026-09-19 06:11:44.82648692 +0000 UTC m=+1850.437166942"
Sep 19 06:11:47.748819 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:47.748775    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:11:50.821518 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:50.821481    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:11:58.880528 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:58.880491    2581 generic.go:356] "Generic (PLEG): container finished" podID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerID="0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b" exitCode=0
Sep 19 06:11:58.880935 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:58.880564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" event={"ID":"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd","Type":"ContainerDied","Data":"0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"}
Sep 19 06:11:58.880935 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:58.880923    2581 scope.go:117] "RemoveContainer" containerID="0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"
Sep 19 06:11:58.882002 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:58.881973    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:11:58.882085 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:58.882069    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:11:58.882133 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:58.882100    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 19 06:11:59.887729 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:59.887698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" event={"ID":"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd","Type":"ContainerStarted","Data":"33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de"}
Sep 19 06:11:59.904817 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:59.904771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:11:59.905233 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:59.905201    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 06:11:59.905505 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:11:59.905473    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 06:12:00.905202 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:00.905160    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 06:12:00.905698 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:00.905234    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 06:12:01.904672 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:01.904638    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 06:12:01.904961 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:01.904932    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 06:12:02.905429 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:02.905385    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 06:12:02.905922 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:02.905452    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 06:12:03.905570 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:03.905113    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 06:12:03.905570 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:03.905178    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 06:12:04.904850 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:04.904815    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 06:12:04.905241 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:04.905210    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 06:12:05.908576 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:05.908545    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:12:05.909050 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:05.908957    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:12:05.909715 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:05.909694    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:12:24.749563 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.749528    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"]
Sep 19 06:12:24.753559 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.753541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.757812 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.757789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7142e3\"/\"default-dockercfg-kgjl8\""
Sep 19 06:12:24.757812 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.757801    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7142e3\"/\"ipp-route-7142e3-kserve-self-signed-certs\""
Sep 19 06:12:24.758010 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.757840    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7142e3\"/\"kube-root-ca.crt\""
Sep 19 06:12:24.758010 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.757907    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7142e3\"/\"openshift-service-ca.crt\""
Sep 19 06:12:24.761771 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.761743    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"]
Sep 19 06:12:24.882851 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.882822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-tmp-dir\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.883018 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.882863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-model-cache\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.883018 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.882891    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48812b9e-5815-4e98-a6b6-04f497e0ad51-tls-certs\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.883018 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.882966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-dshm\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.883018 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.883004    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-home\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.883196 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.883050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlvsw\" (UniqueName: \"kubernetes.io/projected/48812b9e-5815-4e98-a6b6-04f497e0ad51-kube-api-access-qlvsw\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984341 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-tmp-dir\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-model-cache\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48812b9e-5815-4e98-a6b6-04f497e0ad51-tls-certs\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-dshm\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-home\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984525 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlvsw\" (UniqueName: \"kubernetes.io/projected/48812b9e-5815-4e98-a6b6-04f497e0ad51-kube-api-access-qlvsw\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984790 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-model-cache\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984853 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-tmp-dir\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.984907 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.984889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-home\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.986836 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.986814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-dshm\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.987145 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.987120    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48812b9e-5815-4e98-a6b6-04f497e0ad51-tls-certs\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:24.992138 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:24.992091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlvsw\" (UniqueName: \"kubernetes.io/projected/48812b9e-5815-4e98-a6b6-04f497e0ad51-kube-api-access-qlvsw\") pod \"ipp-route-7142e3-kserve-6944ff94c-rwn66\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") " pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:25.064884 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:25.064854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:25.340389 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:25.340365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"]
Sep 19 06:12:25.342275 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:12:25.342249    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48812b9e_5815_4e98_a6b6_04f497e0ad51.slice/crio-2533b17727edb53cdd1ac20ed6aeaf9e343a591c1779ac6222c9b217aff0949f WatchSource:0}: Error finding container 2533b17727edb53cdd1ac20ed6aeaf9e343a591c1779ac6222c9b217aff0949f: Status 404 returned error can't find the container with id 2533b17727edb53cdd1ac20ed6aeaf9e343a591c1779ac6222c9b217aff0949f
Sep 19 06:12:26.032952 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:26.032913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66" event={"ID":"48812b9e-5815-4e98-a6b6-04f497e0ad51","Type":"ContainerStarted","Data":"59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d"}
Sep 19 06:12:26.033357 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:26.032960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66" event={"ID":"48812b9e-5815-4e98-a6b6-04f497e0ad51","Type":"ContainerStarted","Data":"2533b17727edb53cdd1ac20ed6aeaf9e343a591c1779ac6222c9b217aff0949f"}
Sep 19 06:12:26.049532 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:26.049482    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66" podStartSLOduration=2.049467698 podStartE2EDuration="2.049467698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:12:26.047635792 +0000 UTC m=+1891.658315786" watchObservedRunningTime="2026-09-19 06:12:26.049467698 +0000 UTC m=+1891.660147661"
Sep 19 06:12:28.742271 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:28.742232    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:12:35.065847 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:35.065812    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:35.066351 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:35.065859    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:35.078666 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:35.078640    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:35.091272 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:35.091250    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:12:45.351885 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:45.351845    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:12:51.158384 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:51.158350    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z_1b18f557-0f00-4d84-a6d7-a7849f273e78/payload-processing/0.log"
Sep 19 06:12:58.602499 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.602468    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"]
Sep 19 06:12:58.606315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.606288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.608839 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.608816    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4eb285\""
Sep 19 06:12:58.608971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.608951    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4eb285-dockercfg-pshxd\""
Sep 19 06:12:58.615061 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.615020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"]
Sep 19 06:12:58.669847 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.669812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec462db-d2df-41ac-bfca-f17f7dc271ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.669972 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.669902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xvn8\" (UniqueName: \"kubernetes.io/projected/aec462db-d2df-41ac-bfca-f17f7dc271ba-kube-api-access-9xvn8\") pod \"payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.771089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.771028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xvn8\" (UniqueName: \"kubernetes.io/projected/aec462db-d2df-41ac-bfca-f17f7dc271ba-kube-api-access-9xvn8\") pod \"payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.771273 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.771107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec462db-d2df-41ac-bfca-f17f7dc271ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.771759 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.771741    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec462db-d2df-41ac-bfca-f17f7dc271ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.778916 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.778883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xvn8\" (UniqueName: \"kubernetes.io/projected/aec462db-d2df-41ac-bfca-f17f7dc271ba-kube-api-access-9xvn8\") pod \"payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:58.917594 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:58.917509    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:12:59.190949 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:12:59.190915    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"]
Sep 19 06:12:59.193776 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:12:59.193746    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec462db_d2df_41ac_bfca_f17f7dc271ba.slice/crio-914661bd38ffa96ff717d4dac21d99d696df7f3503fd40075fce7ec59f393e1e WatchSource:0}: Error finding container 914661bd38ffa96ff717d4dac21d99d696df7f3503fd40075fce7ec59f393e1e: Status 404 returned error can't find the container with id 914661bd38ffa96ff717d4dac21d99d696df7f3503fd40075fce7ec59f393e1e
Sep 19 06:13:00.201108 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:00.201066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj" event={"ID":"aec462db-d2df-41ac-bfca-f17f7dc271ba","Type":"ContainerStarted","Data":"9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0"}
Sep 19 06:13:00.201522 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:00.201130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj" event={"ID":"aec462db-d2df-41ac-bfca-f17f7dc271ba","Type":"ContainerStarted","Data":"914661bd38ffa96ff717d4dac21d99d696df7f3503fd40075fce7ec59f393e1e"}
Sep 19 06:13:00.201522 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:00.201247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:13:00.217856 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:00.217801    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj" podStartSLOduration=2.217782299 podStartE2EDuration="2.217782299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:13:00.215407591 +0000 UTC m=+1925.826087572" watchObservedRunningTime="2026-09-19 06:13:00.217782299 +0000 UTC m=+1925.828462260"
Sep 19 06:13:06.548939 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:06.548905    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:13:11.208541 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:11.208505    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:13:27.347087 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.347027    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"]
Sep 19 06:13:27.347696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.347307    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj" podUID="aec462db-d2df-41ac-bfca-f17f7dc271ba" containerName="payload-processing" containerID="cri-o://9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0" gracePeriod=30
Sep 19 06:13:27.617250 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.617225    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:13:27.733707 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.733680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec462db-d2df-41ac-bfca-f17f7dc271ba-plugins-config-volume\") pod \"aec462db-d2df-41ac-bfca-f17f7dc271ba\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") "
Sep 19 06:13:27.733866 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.733738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xvn8\" (UniqueName: \"kubernetes.io/projected/aec462db-d2df-41ac-bfca-f17f7dc271ba-kube-api-access-9xvn8\") pod \"aec462db-d2df-41ac-bfca-f17f7dc271ba\" (UID: \"aec462db-d2df-41ac-bfca-f17f7dc271ba\") "
Sep 19 06:13:27.734091 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.734064    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec462db-d2df-41ac-bfca-f17f7dc271ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aec462db-d2df-41ac-bfca-f17f7dc271ba" (UID: "aec462db-d2df-41ac-bfca-f17f7dc271ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:13:27.735962 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.735931    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec462db-d2df-41ac-bfca-f17f7dc271ba-kube-api-access-9xvn8" (OuterVolumeSpecName: "kube-api-access-9xvn8") pod "aec462db-d2df-41ac-bfca-f17f7dc271ba" (UID: "aec462db-d2df-41ac-bfca-f17f7dc271ba"). InnerVolumeSpecName "kube-api-access-9xvn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:13:27.834769 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.834735    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec462db-d2df-41ac-bfca-f17f7dc271ba-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:13:27.834769 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:27.834763    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xvn8\" (UniqueName: \"kubernetes.io/projected/aec462db-d2df-41ac-bfca-f17f7dc271ba-kube-api-access-9xvn8\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:13:28.348951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.348914    2581 generic.go:356] "Generic (PLEG): container finished" podID="aec462db-d2df-41ac-bfca-f17f7dc271ba" containerID="9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0" exitCode=0
Sep 19 06:13:28.348951 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.348951    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"
Sep 19 06:13:28.349439 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.348986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj" event={"ID":"aec462db-d2df-41ac-bfca-f17f7dc271ba","Type":"ContainerDied","Data":"9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0"}
Sep 19 06:13:28.349439 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.349062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj" event={"ID":"aec462db-d2df-41ac-bfca-f17f7dc271ba","Type":"ContainerDied","Data":"914661bd38ffa96ff717d4dac21d99d696df7f3503fd40075fce7ec59f393e1e"}
Sep 19 06:13:28.349439 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.349087    2581 scope.go:117] "RemoveContainer" containerID="9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0"
Sep 19 06:13:28.358790 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.358766    2581 scope.go:117] "RemoveContainer" containerID="9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0"
Sep 19 06:13:28.359079 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:13:28.359058    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0\": container with ID starting with 9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0 not found: ID does not exist" containerID="9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0"
Sep 19 06:13:28.359153 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.359091    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0"} err="failed to get container status \"9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0\": rpc error: code = NotFound desc = could not find container \"9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0\": container with ID starting with 9df2f619f29261e2876050cf5abdc6fe8b8d52fac0f9721f43ba7d02404365c0 not found: ID does not exist"
Sep 19 06:13:28.370194 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.370166    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"]
Sep 19 06:13:28.373744 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.373724    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4eb285-b5d787fdc-mvnmj"]
Sep 19 06:13:28.476934 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:28.476905    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec462db-d2df-41ac-bfca-f17f7dc271ba" path="/var/lib/kubelet/pods/aec462db-d2df-41ac-bfca-f17f7dc271ba/volumes"
Sep 19 06:13:31.639153 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:31.639117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:13:51.238510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:51.238420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:13:52.929751 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:52.929718    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"]
Sep 19 06:13:52.930181 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:52.929994    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z" podUID="1b18f557-0f00-4d84-a6d7-a7849f273e78" containerName="payload-processing" containerID="cri-o://402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e" gracePeriod=30
Sep 19 06:13:53.191161 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.191093    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:13:53.267665 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.267633    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b18f557-0f00-4d84-a6d7-a7849f273e78-plugins-config-volume\") pod \"1b18f557-0f00-4d84-a6d7-a7849f273e78\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") "
Sep 19 06:13:53.267835 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.267733    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqsqt\" (UniqueName: \"kubernetes.io/projected/1b18f557-0f00-4d84-a6d7-a7849f273e78-kube-api-access-xqsqt\") pod \"1b18f557-0f00-4d84-a6d7-a7849f273e78\" (UID: \"1b18f557-0f00-4d84-a6d7-a7849f273e78\") "
Sep 19 06:13:53.268141 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.268110    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b18f557-0f00-4d84-a6d7-a7849f273e78-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b18f557-0f00-4d84-a6d7-a7849f273e78" (UID: "1b18f557-0f00-4d84-a6d7-a7849f273e78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:13:53.270531 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.270509    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b18f557-0f00-4d84-a6d7-a7849f273e78-kube-api-access-xqsqt" (OuterVolumeSpecName: "kube-api-access-xqsqt") pod "1b18f557-0f00-4d84-a6d7-a7849f273e78" (UID: "1b18f557-0f00-4d84-a6d7-a7849f273e78"). InnerVolumeSpecName "kube-api-access-xqsqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:13:53.369246 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.369205    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqsqt\" (UniqueName: \"kubernetes.io/projected/1b18f557-0f00-4d84-a6d7-a7849f273e78-kube-api-access-xqsqt\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:13:53.369246 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.369239    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b18f557-0f00-4d84-a6d7-a7849f273e78-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:13:53.473786 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.473687    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b18f557-0f00-4d84-a6d7-a7849f273e78" containerID="402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e" exitCode=0
Sep 19 06:13:53.473786 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.473746    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"
Sep 19 06:13:53.473786 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.473770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z" event={"ID":"1b18f557-0f00-4d84-a6d7-a7849f273e78","Type":"ContainerDied","Data":"402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e"}
Sep 19 06:13:53.474099 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.473812    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z" event={"ID":"1b18f557-0f00-4d84-a6d7-a7849f273e78","Type":"ContainerDied","Data":"5ec466b0ce165f3b81f576606471588e12a163b8943cf1c4fd6eb12541d8d2e6"}
Sep 19 06:13:53.474099 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.473837    2581 scope.go:117] "RemoveContainer" containerID="402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e"
Sep 19 06:13:53.490595 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.490569    2581 scope.go:117] "RemoveContainer" containerID="402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e"
Sep 19 06:13:53.490919 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:13:53.490896    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e\": container with ID starting with 402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e not found: ID does not exist" containerID="402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e"
Sep 19 06:13:53.491015 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.490929    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e"} err="failed to get container status \"402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e\": rpc error: code = NotFound desc = could not find container \"402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e\": container with ID starting with 402c4b02565120c792ec4168d0590cabc8a9683701e08f354669483d2111a85e not found: ID does not exist"
Sep 19 06:13:53.501481 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.501457    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"]
Sep 19 06:13:53.505041 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:53.505005    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7142e3-7fc854f565-nfd9z"]
Sep 19 06:13:54.479956 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:54.479905    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b18f557-0f00-4d84-a6d7-a7849f273e78" path="/var/lib/kubelet/pods/1b18f557-0f00-4d84-a6d7-a7849f273e78/volumes"
Sep 19 06:13:58.451314 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:13:58.451275    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"]
Sep 19 06:14:03.039138 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:03.039090    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:14:13.473563 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.473507    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66" podUID="48812b9e-5815-4e98-a6b6-04f497e0ad51" containerName="main" containerID="cri-o://59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d" gracePeriod=45
Sep 19 06:14:13.729356 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.729298    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:14:13.855448 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855411    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48812b9e-5815-4e98-a6b6-04f497e0ad51-tls-certs\") pod \"48812b9e-5815-4e98-a6b6-04f497e0ad51\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") "
Sep 19 06:14:13.855617 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855491    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-tmp-dir\") pod \"48812b9e-5815-4e98-a6b6-04f497e0ad51\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") "
Sep 19 06:14:13.855617 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855582    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-dshm\") pod \"48812b9e-5815-4e98-a6b6-04f497e0ad51\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") "
Sep 19 06:14:13.855691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855628    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-model-cache\") pod \"48812b9e-5815-4e98-a6b6-04f497e0ad51\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") "
Sep 19 06:14:13.855691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855653    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-home\") pod \"48812b9e-5815-4e98-a6b6-04f497e0ad51\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") "
Sep 19 06:14:13.855785 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855687    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlvsw\" (UniqueName: \"kubernetes.io/projected/48812b9e-5815-4e98-a6b6-04f497e0ad51-kube-api-access-qlvsw\") pod \"48812b9e-5815-4e98-a6b6-04f497e0ad51\" (UID: \"48812b9e-5815-4e98-a6b6-04f497e0ad51\") "
Sep 19 06:14:13.855785 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855731    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48812b9e-5815-4e98-a6b6-04f497e0ad51" (UID: "48812b9e-5815-4e98-a6b6-04f497e0ad51"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:14:13.855931 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855906    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-home" (OuterVolumeSpecName: "home") pod "48812b9e-5815-4e98-a6b6-04f497e0ad51" (UID: "48812b9e-5815-4e98-a6b6-04f497e0ad51"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:14:13.856005 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.855936    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-model-cache" (OuterVolumeSpecName: "model-cache") pod "48812b9e-5815-4e98-a6b6-04f497e0ad51" (UID: "48812b9e-5815-4e98-a6b6-04f497e0ad51"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:14:13.856132 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.856114    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:13.856186 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.856137    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:13.856186 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.856150    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:13.857872 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.857852    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-dshm" (OuterVolumeSpecName: "dshm") pod "48812b9e-5815-4e98-a6b6-04f497e0ad51" (UID: "48812b9e-5815-4e98-a6b6-04f497e0ad51"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:14:13.857958 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.857935    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48812b9e-5815-4e98-a6b6-04f497e0ad51-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48812b9e-5815-4e98-a6b6-04f497e0ad51" (UID: "48812b9e-5815-4e98-a6b6-04f497e0ad51"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:14:13.858046 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.858010    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48812b9e-5815-4e98-a6b6-04f497e0ad51-kube-api-access-qlvsw" (OuterVolumeSpecName: "kube-api-access-qlvsw") pod "48812b9e-5815-4e98-a6b6-04f497e0ad51" (UID: "48812b9e-5815-4e98-a6b6-04f497e0ad51"). InnerVolumeSpecName "kube-api-access-qlvsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:14:13.957574 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.957543    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48812b9e-5815-4e98-a6b6-04f497e0ad51-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:13.957574 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.957569    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48812b9e-5815-4e98-a6b6-04f497e0ad51-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:13.957574 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:13.957579    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlvsw\" (UniqueName: \"kubernetes.io/projected/48812b9e-5815-4e98-a6b6-04f497e0ad51-kube-api-access-qlvsw\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:14.580016 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.579981    2581 generic.go:356] "Generic (PLEG): container finished" podID="48812b9e-5815-4e98-a6b6-04f497e0ad51" containerID="59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d" exitCode=0
Sep 19 06:14:14.580408 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.580021    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"
Sep 19 06:14:14.580408 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.580069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66" event={"ID":"48812b9e-5815-4e98-a6b6-04f497e0ad51","Type":"ContainerDied","Data":"59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d"}
Sep 19 06:14:14.580408 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.580115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66" event={"ID":"48812b9e-5815-4e98-a6b6-04f497e0ad51","Type":"ContainerDied","Data":"2533b17727edb53cdd1ac20ed6aeaf9e343a591c1779ac6222c9b217aff0949f"}
Sep 19 06:14:14.580408 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.580137    2581 scope.go:117] "RemoveContainer" containerID="59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d"
Sep 19 06:14:14.591176 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.591108    2581 scope.go:117] "RemoveContainer" containerID="59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d"
Sep 19 06:14:14.591550 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:14:14.591520    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d\": container with ID starting with 59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d not found: ID does not exist" containerID="59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d"
Sep 19 06:14:14.591655 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.591549    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d"} err="failed to get container status \"59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d\": rpc error: code = NotFound desc = could not find container \"59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d\": container with ID starting with 59245e5ffd126f65415b5775540897e9caf9f178c2021e8d60b55d4a9b41229d not found: ID does not exist"
Sep 19 06:14:14.596124 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.596100    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"]
Sep 19 06:14:14.597913 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:14.597888    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-7142e3/ipp-route-7142e3-kserve-6944ff94c-rwn66"]
Sep 19 06:14:16.477909 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:16.477875    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48812b9e-5815-4e98-a6b6-04f497e0ad51" path="/var/lib/kubelet/pods/48812b9e-5815-4e98-a6b6-04f497e0ad51/volumes"
Sep 19 06:14:52.484319 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.484282    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"]
Sep 19 06:14:52.484993 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.484619    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" podUID="00a89c41-8388-44c5-8b9d-00bfbea03f70" containerName="maas-api" containerID="cri-o://c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21" gracePeriod=30
Sep 19 06:14:52.743225 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.743165    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:14:52.767488 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.767437    2581 generic.go:356] "Generic (PLEG): container finished" podID="00a89c41-8388-44c5-8b9d-00bfbea03f70" containerID="c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21" exitCode=0
Sep 19 06:14:52.767818 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.767781    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" event={"ID":"00a89c41-8388-44c5-8b9d-00bfbea03f70","Type":"ContainerDied","Data":"c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21"}
Sep 19 06:14:52.768236 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.768214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56" event={"ID":"00a89c41-8388-44c5-8b9d-00bfbea03f70","Type":"ContainerDied","Data":"4288a6fe682dada049468c82692e76059d5e61c626714a330da0d4f8da7d595d"}
Sep 19 06:14:52.768467 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.768004    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"
Sep 19 06:14:52.768911 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.768319    2581 scope.go:117] "RemoveContainer" containerID="c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21"
Sep 19 06:14:52.778221 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.778195    2581 scope.go:117] "RemoveContainer" containerID="c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21"
Sep 19 06:14:52.778506 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:14:52.778485    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21\": container with ID starting with c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21 not found: ID does not exist" containerID="c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21"
Sep 19 06:14:52.778572 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.778514    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21"} err="failed to get container status \"c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21\": rpc error: code = NotFound desc = could not find container \"c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21\": container with ID starting with c9f3dcc10b25e0f5ae64cb6180d923c60517668a69bdf8ef1260cf66c358fe21 not found: ID does not exist"
Sep 19 06:14:52.898405 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.898369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf4qg\" (UniqueName: \"kubernetes.io/projected/00a89c41-8388-44c5-8b9d-00bfbea03f70-kube-api-access-gf4qg\") pod \"00a89c41-8388-44c5-8b9d-00bfbea03f70\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") "
Sep 19 06:14:52.898597 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.898463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls\") pod \"00a89c41-8388-44c5-8b9d-00bfbea03f70\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") "
Sep 19 06:14:52.898597 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.898531    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00a89c41-8388-44c5-8b9d-00bfbea03f70-openshift-service-ca\") pod \"00a89c41-8388-44c5-8b9d-00bfbea03f70\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") "
Sep 19 06:14:52.898597 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.898572    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-metrics-certs\") pod \"00a89c41-8388-44c5-8b9d-00bfbea03f70\" (UID: \"00a89c41-8388-44c5-8b9d-00bfbea03f70\") "
Sep 19 06:14:52.899973 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.899926    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a89c41-8388-44c5-8b9d-00bfbea03f70-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00a89c41-8388-44c5-8b9d-00bfbea03f70" (UID: "00a89c41-8388-44c5-8b9d-00bfbea03f70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:14:52.901721 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.901650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "00a89c41-8388-44c5-8b9d-00bfbea03f70" (UID: "00a89c41-8388-44c5-8b9d-00bfbea03f70"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:14:52.901881 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.901849    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a89c41-8388-44c5-8b9d-00bfbea03f70-kube-api-access-gf4qg" (OuterVolumeSpecName: "kube-api-access-gf4qg") pod "00a89c41-8388-44c5-8b9d-00bfbea03f70" (UID: "00a89c41-8388-44c5-8b9d-00bfbea03f70"). InnerVolumeSpecName "kube-api-access-gf4qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:14:52.902490 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.902430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00a89c41-8388-44c5-8b9d-00bfbea03f70" (UID: "00a89c41-8388-44c5-8b9d-00bfbea03f70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:14:52.999395 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.999330    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf4qg\" (UniqueName: \"kubernetes.io/projected/00a89c41-8388-44c5-8b9d-00bfbea03f70-kube-api-access-gf4qg\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:52.999395 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.999356    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:52.999395 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.999365    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00a89c41-8388-44c5-8b9d-00bfbea03f70-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:52.999395 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:52.999373    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00a89c41-8388-44c5-8b9d-00bfbea03f70-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:14:53.093678 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:53.093644    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"]
Sep 19 06:14:53.097800 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:53.097775    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ebb0f4-6955d785d-dbw56"]
Sep 19 06:14:54.477501 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:54.477463    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00a89c41-8388-44c5-8b9d-00bfbea03f70" path="/var/lib/kubelet/pods/00a89c41-8388-44c5-8b9d-00bfbea03f70/volumes"
Sep 19 06:14:56.041147 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:56.041110    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:14:58.974575 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:58.974538    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"]
Sep 19 06:14:58.975024 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:14:58.974811    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy" containerID="cri-o://33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de" gracePeriod=30
Sep 19 06:15:00.137929 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.137892    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"]
Sep 19 06:15:00.138315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138256    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00a89c41-8388-44c5-8b9d-00bfbea03f70" containerName="maas-api"
Sep 19 06:15:00.138315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138268    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00a89c41-8388-44c5-8b9d-00bfbea03f70" containerName="maas-api"
Sep 19 06:15:00.138315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138285    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48812b9e-5815-4e98-a6b6-04f497e0ad51" containerName="main"
Sep 19 06:15:00.138315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138291    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="48812b9e-5815-4e98-a6b6-04f497e0ad51" containerName="main"
Sep 19 06:15:00.138315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138303    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec462db-d2df-41ac-bfca-f17f7dc271ba" containerName="payload-processing"
Sep 19 06:15:00.138315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138310    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec462db-d2df-41ac-bfca-f17f7dc271ba" containerName="payload-processing"
Sep 19 06:15:00.138675 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138321    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b18f557-0f00-4d84-a6d7-a7849f273e78" containerName="payload-processing"
Sep 19 06:15:00.138675 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138327    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b18f557-0f00-4d84-a6d7-a7849f273e78" containerName="payload-processing"
Sep 19 06:15:00.138675 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138378    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b18f557-0f00-4d84-a6d7-a7849f273e78" containerName="payload-processing"
Sep 19 06:15:00.138675 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138385    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec462db-d2df-41ac-bfca-f17f7dc271ba" containerName="payload-processing"
Sep 19 06:15:00.138675 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138427    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00a89c41-8388-44c5-8b9d-00bfbea03f70" containerName="maas-api"
Sep 19 06:15:00.138675 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.138434    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="48812b9e-5815-4e98-a6b6-04f497e0ad51" containerName="main"
Sep 19 06:15:00.143310 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.143289    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"]
Sep 19 06:15:00.143415 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.143399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"
Sep 19 06:15:00.145825 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.145806    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\""
Sep 19 06:15:00.146360 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.146341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"
Sep 19 06:15:00.148788 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.148766    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"]
Sep 19 06:15:00.153606 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.153578    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"]
Sep 19 06:15:00.157307 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.157286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"
Sep 19 06:15:00.165225 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.165206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"
Sep 19 06:15:00.438862 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.438695    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"]
Sep 19 06:15:00.440897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.440876    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"]
Sep 19 06:15:00.442242 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:15:00.442214    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2f6f83_521f_446a_b0ac_8591c8634029.slice/crio-d18fe5bde2d5df1d0b299d625b97295d91f4b6a3a4fe2590fb602455c934c25a WatchSource:0}: Error finding container d18fe5bde2d5df1d0b299d625b97295d91f4b6a3a4fe2590fb602455c934c25a: Status 404 returned error can't find the container with id d18fe5bde2d5df1d0b299d625b97295d91f4b6a3a4fe2590fb602455c934c25a
Sep 19 06:15:00.443168 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:15:00.443144    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1453eba6_e084_409e_b6a9_aa5174f2cff2.slice/crio-cee518809de2207b90dc1d2fab1783ff6234be3f4d18d91b0fa9b1ff52dea536 WatchSource:0}: Error finding container cee518809de2207b90dc1d2fab1783ff6234be3f4d18d91b0fa9b1ff52dea536: Status 404 returned error can't find the container with id cee518809de2207b90dc1d2fab1783ff6234be3f4d18d91b0fa9b1ff52dea536
Sep 19 06:15:00.808085 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.808027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99" event={"ID":"fd2f6f83-521f-446a-b0ac-8591c8634029","Type":"ContainerStarted","Data":"0b58233179c470d8b47d061e80070db3d5479aa66e59ff56d65ad41b5a07a872"}
Sep 19 06:15:00.808259 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.808118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99" event={"ID":"fd2f6f83-521f-446a-b0ac-8591c8634029","Type":"ContainerStarted","Data":"d18fe5bde2d5df1d0b299d625b97295d91f4b6a3a4fe2590fb602455c934c25a"}
Sep 19 06:15:00.809528 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.809503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5" event={"ID":"1453eba6-e084-409e-b6a9-aa5174f2cff2","Type":"ContainerStarted","Data":"0a80b88771c7695b62ac4cf720891923f4d1dc9fdca9e2f66178dc4c7a48666c"}
Sep 19 06:15:00.809641 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.809552    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5" event={"ID":"1453eba6-e084-409e-b6a9-aa5174f2cff2","Type":"ContainerStarted","Data":"cee518809de2207b90dc1d2fab1783ff6234be3f4d18d91b0fa9b1ff52dea536"}
Sep 19 06:15:00.822476 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.822429    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99" podStartSLOduration=0.822416321 podStartE2EDuration="822.416321ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:15:00.821545117 +0000 UTC m=+2046.432225116" watchObservedRunningTime="2026-09-19 06:15:00.822416321 +0000 UTC m=+2046.433096278"
Sep 19 06:15:00.837909 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:00.837867    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5" podStartSLOduration=0.837855024 podStartE2EDuration="837.855024ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:15:00.83693935 +0000 UTC m=+2046.447619352" watchObservedRunningTime="2026-09-19 06:15:00.837855024 +0000 UTC m=+2046.448535038"
Sep 19 06:15:01.815808 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:01.815769    2581 generic.go:356] "Generic (PLEG): container finished" podID="1453eba6-e084-409e-b6a9-aa5174f2cff2" containerID="0a80b88771c7695b62ac4cf720891923f4d1dc9fdca9e2f66178dc4c7a48666c" exitCode=0
Sep 19 06:15:01.816241 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:01.815855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5" event={"ID":"1453eba6-e084-409e-b6a9-aa5174f2cff2","Type":"ContainerDied","Data":"0a80b88771c7695b62ac4cf720891923f4d1dc9fdca9e2f66178dc4c7a48666c"}
Sep 19 06:15:01.817510 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:01.817486    2581 generic.go:356] "Generic (PLEG): container finished" podID="fd2f6f83-521f-446a-b0ac-8591c8634029" containerID="0b58233179c470d8b47d061e80070db3d5479aa66e59ff56d65ad41b5a07a872" exitCode=0
Sep 19 06:15:01.817614 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:01.817522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99" event={"ID":"fd2f6f83-521f-446a-b0ac-8591c8634029","Type":"ContainerDied","Data":"0b58233179c470d8b47d061e80070db3d5479aa66e59ff56d65ad41b5a07a872"}
Sep 19 06:15:02.953953 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:02.953929    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"
Sep 19 06:15:02.983722 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:02.983697    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"
Sep 19 06:15:03.829120 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:03.829087    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"
Sep 19 06:15:03.829120 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:03.829100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99" event={"ID":"fd2f6f83-521f-446a-b0ac-8591c8634029","Type":"ContainerDied","Data":"d18fe5bde2d5df1d0b299d625b97295d91f4b6a3a4fe2590fb602455c934c25a"}
Sep 19 06:15:03.829320 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:03.829144    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d18fe5bde2d5df1d0b299d625b97295d91f4b6a3a4fe2590fb602455c934c25a"
Sep 19 06:15:03.830464 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:03.830438    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5"
Sep 19 06:15:03.830586 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:03.830474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829975-sw5b5" event={"ID":"1453eba6-e084-409e-b6a9-aa5174f2cff2","Type":"ContainerDied","Data":"cee518809de2207b90dc1d2fab1783ff6234be3f4d18d91b0fa9b1ff52dea536"}
Sep 19 06:15:03.830586 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:03.830509    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cee518809de2207b90dc1d2fab1783ff6234be3f4d18d91b0fa9b1ff52dea536"
Sep 19 06:15:04.235928 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.235905    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:15:04.400773 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400695    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-podinfo\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.400773 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-socket\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.400773 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-certs\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400789    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-credential-socket\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400806    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-envoy\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-token\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400900    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-data\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istiod-ca-cert\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.400964    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7wj2\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-kube-api-access-k7wj2\") pod \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\" (UID: \"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd\") "
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401091    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401110    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401121    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401344    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401364    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-workload-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401378    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-credential-socket\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401437    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:15:04.401455 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.401435    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:04.403141 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.403117    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:15:04.403254 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.403234    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:04.403625 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.403594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:04.403721 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.403623    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-kube-api-access-k7wj2" (OuterVolumeSpecName: "kube-api-access-k7wj2") pod "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" (UID: "e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd"). InnerVolumeSpecName "kube-api-access-k7wj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:04.502512 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.502490    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-podinfo\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.502512 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.502512    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-envoy\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.502658 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.502521    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-token\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.502658 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.502529    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istio-data\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.502658 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.502536    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-istiod-ca-cert\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.502658 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.502544    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7wj2\" (UniqueName: \"kubernetes.io/projected/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd-kube-api-access-k7wj2\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:04.837103 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.837063    2581 generic.go:356] "Generic (PLEG): container finished" podID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerID="33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de" exitCode=0
Sep 19 06:15:04.837103 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.837103    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"
Sep 19 06:15:04.837318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.837138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" event={"ID":"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd","Type":"ContainerDied","Data":"33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de"}
Sep 19 06:15:04.837318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.837183    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj" event={"ID":"e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd","Type":"ContainerDied","Data":"2b21d86002d2027db151fa65b6de50c3812e8722e8fe905dadc88ba973e3ff76"}
Sep 19 06:15:04.837318 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.837207    2581 scope.go:117] "RemoveContainer" containerID="33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de"
Sep 19 06:15:04.846440 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.846421    2581 scope.go:117] "RemoveContainer" containerID="0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"
Sep 19 06:15:04.854514 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.854490    2581 scope.go:117] "RemoveContainer" containerID="33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de"
Sep 19 06:15:04.854638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.854607    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"]
Sep 19 06:15:04.854802 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:15:04.854783    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de\": container with ID starting with 33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de not found: ID does not exist" containerID="33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de"
Sep 19 06:15:04.854841 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.854809    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de"} err="failed to get container status \"33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de\": rpc error: code = NotFound desc = could not find container \"33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de\": container with ID starting with 33d35ad428dbe21fc1cc48b5fbe48bb733159240a9e4eba7848a5967895f33de not found: ID does not exist"
Sep 19 06:15:04.854841 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.854826    2581 scope.go:117] "RemoveContainer" containerID="0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"
Sep 19 06:15:04.855093 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:15:04.855078    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b\": container with ID starting with 0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b not found: ID does not exist" containerID="0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"
Sep 19 06:15:04.855149 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.855097    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b"} err="failed to get container status \"0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b\": rpc error: code = NotFound desc = could not find container \"0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b\": container with ID starting with 0f0c6decbba45bbc31a11b09a044df40154552ee9c3b994c7c6135e8eba6ec3b not found: ID does not exist"
Sep 19 06:15:04.860360 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:04.860330    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-ebb0f4-openshift-default-5bbcbc8678-cnfjj"]
Sep 19 06:15:06.477413 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:06.477379    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" path="/var/lib/kubelet/pods/e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd/volumes"
Sep 19 06:15:09.804231 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:09.804187    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"]
Sep 19 06:15:09.804839 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:09.804490    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9" podUID="1723097d-ff40-4ea0-aaae-c92ee5d747db" containerName="payload-pre-processing" containerID="cri-o://cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df" gracePeriod=30
Sep 19 06:15:10.056237 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.056172    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:15:10.245822 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.245795    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1723097d-ff40-4ea0-aaae-c92ee5d747db-plugins-config-volume\") pod \"1723097d-ff40-4ea0-aaae-c92ee5d747db\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") "
Sep 19 06:15:10.245970 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.245830    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq9n7\" (UniqueName: \"kubernetes.io/projected/1723097d-ff40-4ea0-aaae-c92ee5d747db-kube-api-access-zq9n7\") pod \"1723097d-ff40-4ea0-aaae-c92ee5d747db\" (UID: \"1723097d-ff40-4ea0-aaae-c92ee5d747db\") "
Sep 19 06:15:10.246202 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.246175    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1723097d-ff40-4ea0-aaae-c92ee5d747db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1723097d-ff40-4ea0-aaae-c92ee5d747db" (UID: "1723097d-ff40-4ea0-aaae-c92ee5d747db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:15:10.248170 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.248148    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1723097d-ff40-4ea0-aaae-c92ee5d747db-kube-api-access-zq9n7" (OuterVolumeSpecName: "kube-api-access-zq9n7") pod "1723097d-ff40-4ea0-aaae-c92ee5d747db" (UID: "1723097d-ff40-4ea0-aaae-c92ee5d747db"). InnerVolumeSpecName "kube-api-access-zq9n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:10.346558 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.346487    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1723097d-ff40-4ea0-aaae-c92ee5d747db-plugins-config-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:10.346558 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.346514    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq9n7\" (UniqueName: \"kubernetes.io/projected/1723097d-ff40-4ea0-aaae-c92ee5d747db-kube-api-access-zq9n7\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:10.874663 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.874623    2581 generic.go:356] "Generic (PLEG): container finished" podID="1723097d-ff40-4ea0-aaae-c92ee5d747db" containerID="cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df" exitCode=0
Sep 19 06:15:10.875095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.874673    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"
Sep 19 06:15:10.875095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.874676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9" event={"ID":"1723097d-ff40-4ea0-aaae-c92ee5d747db","Type":"ContainerDied","Data":"cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df"}
Sep 19 06:15:10.875095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.874732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9" event={"ID":"1723097d-ff40-4ea0-aaae-c92ee5d747db","Type":"ContainerDied","Data":"adebc85ce62b8cb7341ad865e26af51d7589e55eeefd0eb45feceb2fb42ca9ad"}
Sep 19 06:15:10.875095 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.874762    2581 scope.go:117] "RemoveContainer" containerID="cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df"
Sep 19 06:15:10.883254 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.883232    2581 scope.go:117] "RemoveContainer" containerID="cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df"
Sep 19 06:15:10.883513 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:15:10.883490    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df\": container with ID starting with cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df not found: ID does not exist" containerID="cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df"
Sep 19 06:15:10.883577 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.883526    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df"} err="failed to get container status \"cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df\": rpc error: code = NotFound desc = could not find container \"cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df\": container with ID starting with cd52fc2f1d0d42307f6f28aa25f268829f821efed3f29cdecf8af7afd45f04df not found: ID does not exist"
Sep 19 06:15:10.895483 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.895455    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"]
Sep 19 06:15:10.900701 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:10.900679    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-888f75-786b648fb6-r8xf9"]
Sep 19 06:15:12.476866 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:12.476832    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1723097d-ff40-4ea0-aaae-c92ee5d747db" path="/var/lib/kubelet/pods/1723097d-ff40-4ea0-aaae-c92ee5d747db/volumes"
Sep 19 06:15:13.049836 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:13.049802    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:15:27.365107 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.364985    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"]
Sep 19 06:15:27.365706 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.365246    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft" podUID="61bf6ba4-212a-4d32-a744-fbc4b032cceb" containerName="maas-api" containerID="cri-o://6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2" gracePeriod=30
Sep 19 06:15:27.391754 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.391728    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"]
Sep 19 06:15:27.396520 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.396491    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-f1aad1-29829975-nsv99"]
Sep 19 06:15:27.616314 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.616252    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:15:27.680266 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.680235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn6ln\" (UniqueName: \"kubernetes.io/projected/61bf6ba4-212a-4d32-a744-fbc4b032cceb-kube-api-access-bn6ln\") pod \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") "
Sep 19 06:15:27.680405 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.680296    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61bf6ba4-212a-4d32-a744-fbc4b032cceb-openshift-service-ca\") pod \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") "
Sep 19 06:15:27.680405 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.680357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-maas-api-tls\") pod \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") "
Sep 19 06:15:27.680405 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.680394    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-metrics-certs\") pod \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\" (UID: \"61bf6ba4-212a-4d32-a744-fbc4b032cceb\") "
Sep 19 06:15:27.680717 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.680695    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61bf6ba4-212a-4d32-a744-fbc4b032cceb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61bf6ba4-212a-4d32-a744-fbc4b032cceb" (UID: "61bf6ba4-212a-4d32-a744-fbc4b032cceb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:15:27.682452 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.682430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61bf6ba4-212a-4d32-a744-fbc4b032cceb-kube-api-access-bn6ln" (OuterVolumeSpecName: "kube-api-access-bn6ln") pod "61bf6ba4-212a-4d32-a744-fbc4b032cceb" (UID: "61bf6ba4-212a-4d32-a744-fbc4b032cceb"). InnerVolumeSpecName "kube-api-access-bn6ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:27.682636 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.682614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "61bf6ba4-212a-4d32-a744-fbc4b032cceb" (UID: "61bf6ba4-212a-4d32-a744-fbc4b032cceb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:15:27.682636 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.682626    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61bf6ba4-212a-4d32-a744-fbc4b032cceb" (UID: "61bf6ba4-212a-4d32-a744-fbc4b032cceb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:15:27.781314 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.781279    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:27.781314 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.781310    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61bf6ba4-212a-4d32-a744-fbc4b032cceb-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:27.781494 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.781325    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn6ln\" (UniqueName: \"kubernetes.io/projected/61bf6ba4-212a-4d32-a744-fbc4b032cceb-kube-api-access-bn6ln\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:27.781494 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.781337    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61bf6ba4-212a-4d32-a744-fbc4b032cceb-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:15:27.956351 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.956266    2581 generic.go:356] "Generic (PLEG): container finished" podID="61bf6ba4-212a-4d32-a744-fbc4b032cceb" containerID="6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2" exitCode=0
Sep 19 06:15:27.956351 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.956309    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"
Sep 19 06:15:27.956521 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.956355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft" event={"ID":"61bf6ba4-212a-4d32-a744-fbc4b032cceb","Type":"ContainerDied","Data":"6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2"}
Sep 19 06:15:27.956521 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.956409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft" event={"ID":"61bf6ba4-212a-4d32-a744-fbc4b032cceb","Type":"ContainerDied","Data":"ae2335acb933c52b3ca5484bc20b059109db232d59f463c382185295709e09f2"}
Sep 19 06:15:27.956521 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.956431    2581 scope.go:117] "RemoveContainer" containerID="6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2"
Sep 19 06:15:27.966056 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.966016    2581 scope.go:117] "RemoveContainer" containerID="6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2"
Sep 19 06:15:27.966311 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:15:27.966293    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2\": container with ID starting with 6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2 not found: ID does not exist" containerID="6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2"
Sep 19 06:15:27.966365 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.966319    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2"} err="failed to get container status \"6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2\": rpc error: code = NotFound desc = could not find container \"6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2\": container with ID starting with 6206da2664a5ed4be1f6157715f2c2488e6cd9b534a41be7802616744bc0c2e2 not found: ID does not exist"
Sep 19 06:15:27.979554 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.979532    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"]
Sep 19 06:15:27.988072 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:27.983312    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-f1aad1-74d8d67b68-pnrft"]
Sep 19 06:15:28.477146 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:28.477115    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61bf6ba4-212a-4d32-a744-fbc4b032cceb" path="/var/lib/kubelet/pods/61bf6ba4-212a-4d32-a744-fbc4b032cceb/volumes"
Sep 19 06:15:28.477522 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:28.477395    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd2f6f83-521f-446a-b0ac-8591c8634029" path="/var/lib/kubelet/pods/fd2f6f83-521f-446a-b0ac-8591c8634029/volumes"
Sep 19 06:15:30.046730 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:30.046693    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:15:39.042622 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:39.042587    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:15:56.566290 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:56.566259    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:15:56.571496 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:56.571475    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:15:56.572758 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:56.572739    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:15:56.577419 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:15:56.577402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:17:02.644395 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:17:02.644357    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:17:53.037998 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:17:53.037956    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:18:02.341114 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:02.341076    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:18:09.944806 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:09.944710    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:18:18.235426 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:18.235395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:18:31.048067 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:31.048016    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:18:39.339624 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:39.339583    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:18:46.252129 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.252083    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"]
Sep 19 06:18:46.252691 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.252319    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj" podUID="f5d370e9-70e2-43d6-8284-416734b7bcb9" containerName="manager" containerID="cri-o://7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035" gracePeriod=10
Sep 19 06:18:46.599900 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.599876    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 06:18:46.729164 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.729134    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsqrx\" (UniqueName: \"kubernetes.io/projected/f5d370e9-70e2-43d6-8284-416734b7bcb9-kube-api-access-nsqrx\") pod \"f5d370e9-70e2-43d6-8284-416734b7bcb9\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") "
Sep 19 06:18:46.729339 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.729217    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d370e9-70e2-43d6-8284-416734b7bcb9-extensions-socket-volume\") pod \"f5d370e9-70e2-43d6-8284-416734b7bcb9\" (UID: \"f5d370e9-70e2-43d6-8284-416734b7bcb9\") "
Sep 19 06:18:46.729615 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.729586    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d370e9-70e2-43d6-8284-416734b7bcb9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f5d370e9-70e2-43d6-8284-416734b7bcb9" (UID: "f5d370e9-70e2-43d6-8284-416734b7bcb9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:18:46.731470 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.731450    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5d370e9-70e2-43d6-8284-416734b7bcb9-kube-api-access-nsqrx" (OuterVolumeSpecName: "kube-api-access-nsqrx") pod "f5d370e9-70e2-43d6-8284-416734b7bcb9" (UID: "f5d370e9-70e2-43d6-8284-416734b7bcb9"). InnerVolumeSpecName "kube-api-access-nsqrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:18:46.829944 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.829910    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsqrx\" (UniqueName: \"kubernetes.io/projected/f5d370e9-70e2-43d6-8284-416734b7bcb9-kube-api-access-nsqrx\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:18:46.829944 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.829939    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d370e9-70e2-43d6-8284-416734b7bcb9-extensions-socket-volume\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:18:46.884904 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.884864    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5d370e9-70e2-43d6-8284-416734b7bcb9" containerID="7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035" exitCode=0
Sep 19 06:18:46.885107 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.884908    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"
Sep 19 06:18:46.885107 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.884961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj" event={"ID":"f5d370e9-70e2-43d6-8284-416734b7bcb9","Type":"ContainerDied","Data":"7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035"}
Sep 19 06:18:46.885107 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.885023    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj" event={"ID":"f5d370e9-70e2-43d6-8284-416734b7bcb9","Type":"ContainerDied","Data":"cdbde243e7a1d174eec88afbf391d9ac4fec9daeefa06aee6103fc4893476f1c"}
Sep 19 06:18:46.885107 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.885069    2581 scope.go:117] "RemoveContainer" containerID="7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035"
Sep 19 06:18:46.894279 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.894262    2581 scope.go:117] "RemoveContainer" containerID="7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035"
Sep 19 06:18:46.894563 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:18:46.894540    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035\": container with ID starting with 7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035 not found: ID does not exist" containerID="7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035"
Sep 19 06:18:46.894655 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.894572    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035"} err="failed to get container status \"7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035\": rpc error: code = NotFound desc = could not find container \"7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035\": container with ID starting with 7fb39eed9adc853bde2a4c5793b1668faecf35b08fc992fb2033c03ac368f035 not found: ID does not exist"
Sep 19 06:18:46.905625 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.905602    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"]
Sep 19 06:18:46.909181 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:46.909160    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dhbsj"]
Sep 19 06:18:48.476710 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:18:48.476675    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5d370e9-70e2-43d6-8284-416734b7bcb9" path="/var/lib/kubelet/pods/f5d370e9-70e2-43d6-8284-416734b7bcb9/volumes"
Sep 19 06:19:46.952975 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.952896    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"]
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953268    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1723097d-ff40-4ea0-aaae-c92ee5d747db" containerName="payload-pre-processing"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953281    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1723097d-ff40-4ea0-aaae-c92ee5d747db" containerName="payload-pre-processing"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953288    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1453eba6-e084-409e-b6a9-aa5174f2cff2" containerName="cleanup"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953294    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1453eba6-e084-409e-b6a9-aa5174f2cff2" containerName="cleanup"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953304    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953311    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953323    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd2f6f83-521f-446a-b0ac-8591c8634029" containerName="cleanup"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953327    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd2f6f83-521f-446a-b0ac-8591c8634029" containerName="cleanup"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953338    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bf6ba4-212a-4d32-a744-fbc4b032cceb" containerName="maas-api"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953343    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bf6ba4-212a-4d32-a744-fbc4b032cceb" containerName="maas-api"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953352    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5d370e9-70e2-43d6-8284-416734b7bcb9" containerName="manager"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953357    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5d370e9-70e2-43d6-8284-416734b7bcb9" containerName="manager"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953408    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="61bf6ba4-212a-4d32-a744-fbc4b032cceb" containerName="maas-api"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953414    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1453eba6-e084-409e-b6a9-aa5174f2cff2" containerName="cleanup"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953422    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5d370e9-70e2-43d6-8284-416734b7bcb9" containerName="manager"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953427    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953432    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd2f6f83-521f-446a-b0ac-8591c8634029" containerName="cleanup"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953439    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy"
Sep 19 06:19:46.953465 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.953444    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1723097d-ff40-4ea0-aaae-c92ee5d747db" containerName="payload-pre-processing"
Sep 19 06:19:46.956251 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.956236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:46.959115 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.959091    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tssrv\""
Sep 19 06:19:46.970523 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:46.970500    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"]
Sep 19 06:19:47.034462 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.034433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33c1058d-19a6-4181-9a8d-534d5eba7d7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2bzdd\" (UID: \"33c1058d-19a6-4181-9a8d-534d5eba7d7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.034589 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.034522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfvnh\" (UniqueName: \"kubernetes.io/projected/33c1058d-19a6-4181-9a8d-534d5eba7d7d-kube-api-access-bfvnh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2bzdd\" (UID: \"33c1058d-19a6-4181-9a8d-534d5eba7d7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.135934 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.135904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfvnh\" (UniqueName: \"kubernetes.io/projected/33c1058d-19a6-4181-9a8d-534d5eba7d7d-kube-api-access-bfvnh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2bzdd\" (UID: \"33c1058d-19a6-4181-9a8d-534d5eba7d7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.136102 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.135994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33c1058d-19a6-4181-9a8d-534d5eba7d7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2bzdd\" (UID: \"33c1058d-19a6-4181-9a8d-534d5eba7d7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.136362 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.136342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33c1058d-19a6-4181-9a8d-534d5eba7d7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2bzdd\" (UID: \"33c1058d-19a6-4181-9a8d-534d5eba7d7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.147144 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.147126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfvnh\" (UniqueName: \"kubernetes.io/projected/33c1058d-19a6-4181-9a8d-534d5eba7d7d-kube-api-access-bfvnh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2bzdd\" (UID: \"33c1058d-19a6-4181-9a8d-534d5eba7d7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.267329 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.267249    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:47.541955 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:47.541878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"]
Sep 19 06:19:47.544579 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:19:47.544552    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c1058d_19a6_4181_9a8d_534d5eba7d7d.slice/crio-5b2258d38a77b0025285beed95947cc49dbe7c28c77e4ba9874289853d52441c WatchSource:0}: Error finding container 5b2258d38a77b0025285beed95947cc49dbe7c28c77e4ba9874289853d52441c: Status 404 returned error can't find the container with id 5b2258d38a77b0025285beed95947cc49dbe7c28c77e4ba9874289853d52441c
Sep 19 06:19:48.159109 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:48.159068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd" event={"ID":"33c1058d-19a6-4181-9a8d-534d5eba7d7d","Type":"ContainerStarted","Data":"67209caeee625c684272bde13b88e6d699ece8288bf4ecfefcf34c34d372dfa7"}
Sep 19 06:19:48.159485 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:48.159130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd" event={"ID":"33c1058d-19a6-4181-9a8d-534d5eba7d7d","Type":"ContainerStarted","Data":"5b2258d38a77b0025285beed95947cc49dbe7c28c77e4ba9874289853d52441c"}
Sep 19 06:19:48.159485 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:48.159205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:19:48.182176 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:48.182130    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd" podStartSLOduration=2.1821155389999998 podStartE2EDuration="2.182115539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:19:48.179224907 +0000 UTC m=+2333.789904911" watchObservedRunningTime="2026-09-19 06:19:48.182115539 +0000 UTC m=+2333.792795504"
Sep 19 06:19:59.166573 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:19:59.166543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2bzdd"
Sep 19 06:20:08.944295 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:08.944260    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:20:17.437267 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:17.437228    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:20:25.935392 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:25.935353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:20:34.039533 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.039494    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"]
Sep 19 06:20:34.039911 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.039844    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy"
Sep 19 06:20:34.039911 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.039860    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cde2ed-f3e9-4071-a87e-3bfdcaeec3fd" containerName="istio-proxy"
Sep 19 06:20:34.043284 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.043260    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.046176 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.046157    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8f62b8ef-kserve-self-signed-certs\""
Sep 19 06:20:34.051970 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.051936    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"]
Sep 19 06:20:34.133825 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.133793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-home\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.133825 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.133826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tls-certs\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.134070 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.133889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tmp-dir\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.134070 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.133949    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-model-cache\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.134070 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.134018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx84n\" (UniqueName: \"kubernetes.io/projected/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-kube-api-access-nx84n\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.134196 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.134080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-dshm\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.234589 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.234551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tmp-dir\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.234774 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.234603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-model-cache\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.234774 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.234646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx84n\" (UniqueName: \"kubernetes.io/projected/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-kube-api-access-nx84n\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.234774 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.234665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-dshm\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.234774 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.234683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-home\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.234774 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.234699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tls-certs\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.235076 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.235020    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tmp-dir\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.235133 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.235061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-model-cache\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.235190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.235169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-home\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.237227 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.237199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-dshm\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.237468 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.237454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tls-certs\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.242670 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.242650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx84n\" (UniqueName: \"kubernetes.io/projected/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-kube-api-access-nx84n\") pod \"e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") " pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.352991 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.352904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:34.624796 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:34.624734    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"]
Sep 19 06:20:34.627363 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:20:34.627334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9e00cd2_cef1_4e6d_89d1_8fc32a410b7f.slice/crio-cf060f3c25df56b80635b16abe34336506068a2dd3ab67b3b426467a5a84bd98 WatchSource:0}: Error finding container cf060f3c25df56b80635b16abe34336506068a2dd3ab67b3b426467a5a84bd98: Status 404 returned error can't find the container with id cf060f3c25df56b80635b16abe34336506068a2dd3ab67b3b426467a5a84bd98
Sep 19 06:20:35.375548 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:35.375511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm" event={"ID":"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f","Type":"ContainerStarted","Data":"a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720"}
Sep 19 06:20:35.375920 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:35.375555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm" event={"ID":"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f","Type":"ContainerStarted","Data":"cf060f3c25df56b80635b16abe34336506068a2dd3ab67b3b426467a5a84bd98"}
Sep 19 06:20:35.393312 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:35.393253    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm" podStartSLOduration=1.393232887 podStartE2EDuration="1.393232887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:20:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:20:35.390445723 +0000 UTC m=+2381.001125710" watchObservedRunningTime="2026-09-19 06:20:35.393232887 +0000 UTC m=+2381.003912852"
Sep 19 06:20:36.645737 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:36.645697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:20:44.353124 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:44.353086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:44.353648 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:44.353167    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:44.365756 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:44.365728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:44.430099 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:44.430075    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:20:47.747122 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.747086    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"]
Sep 19 06:20:47.751063 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.751019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.753714 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.753692    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-8f62b8ef-kserve-self-signed-certs\""
Sep 19 06:20:47.759458 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.758898    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"]
Sep 19 06:20:47.850715 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.850674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptt6k\" (UniqueName: \"kubernetes.io/projected/8bf08e71-b488-4aec-ba51-0eed41f8b422-kube-api-access-ptt6k\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.850897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.850722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-home\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.850897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.850749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-dshm\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.850897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.850837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-tmp-dir\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.850897 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.850874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-model-cache\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.851119 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.850907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf08e71-b488-4aec-ba51-0eed41f8b422-tls-certs\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952159 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-home\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-dshm\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-tmp-dir\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-model-cache\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf08e71-b488-4aec-ba51-0eed41f8b422-tls-certs\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952315 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptt6k\" (UniqueName: \"kubernetes.io/projected/8bf08e71-b488-4aec-ba51-0eed41f8b422-kube-api-access-ptt6k\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952616 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-home\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952666 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-tmp-dir\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.952666 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.952653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-model-cache\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.954728 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.954699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-dshm\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.954979 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.954959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf08e71-b488-4aec-ba51-0eed41f8b422-tls-certs\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:47.959910 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:47.959888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptt6k\" (UniqueName: \"kubernetes.io/projected/8bf08e71-b488-4aec-ba51-0eed41f8b422-kube-api-access-ptt6k\") pod \"e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") " pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:48.062502 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:48.062449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:48.332081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:48.332054    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"]
Sep 19 06:20:48.334110 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:20:48.334085    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bf08e71_b488_4aec_ba51_0eed41f8b422.slice/crio-45aa86eaafeddaa6a22cc1a2e35f0ad56470fdbc997a1a6b9e3ba3573898d824 WatchSource:0}: Error finding container 45aa86eaafeddaa6a22cc1a2e35f0ad56470fdbc997a1a6b9e3ba3573898d824: Status 404 returned error can't find the container with id 45aa86eaafeddaa6a22cc1a2e35f0ad56470fdbc997a1a6b9e3ba3573898d824
Sep 19 06:20:48.438743 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:48.438705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw" event={"ID":"8bf08e71-b488-4aec-ba51-0eed41f8b422","Type":"ContainerStarted","Data":"523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd"}
Sep 19 06:20:48.438910 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:48.438754    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw" event={"ID":"8bf08e71-b488-4aec-ba51-0eed41f8b422","Type":"ContainerStarted","Data":"45aa86eaafeddaa6a22cc1a2e35f0ad56470fdbc997a1a6b9e3ba3573898d824"}
Sep 19 06:20:48.455248 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:48.455197    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw" podStartSLOduration=1.455182702 podStartE2EDuration="1.455182702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:20:48.454193937 +0000 UTC m=+2394.064873960" watchObservedRunningTime="2026-09-19 06:20:48.455182702 +0000 UTC m=+2394.065862666"
Sep 19 06:20:50.643752 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:50.643709    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:20:56.593927 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:56.593891    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:20:56.599242 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:56.599215    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:20:56.602084 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:56.602063    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:20:56.606723 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:56.606707    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9592_95225afd-e021-41c3-8fcb-b99d36b67473/ovn-acl-logging/0.log"
Sep 19 06:20:58.063568 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:58.063535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:58.064078 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:58.063616    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:58.076160 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:58.076139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:20:58.499145 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:20:58.499067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:21:14.083432 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:14.083396    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"]
Sep 19 06:21:14.626501 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:14.626466    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"]
Sep 19 06:21:17.537206 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:17.537174    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:21:27.039205 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:27.039173    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:21:29.104938 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.104889    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm" podUID="f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" containerName="main" containerID="cri-o://a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720" gracePeriod=45
Sep 19 06:21:29.358658 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.358586    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:21:29.403357 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403323    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-dshm\") pod \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") "
Sep 19 06:21:29.403549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403383    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-home\") pod \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") "
Sep 19 06:21:29.403549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-model-cache\") pod \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") "
Sep 19 06:21:29.403549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403446    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx84n\" (UniqueName: \"kubernetes.io/projected/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-kube-api-access-nx84n\") pod \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") "
Sep 19 06:21:29.403549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403466    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tls-certs\") pod \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") "
Sep 19 06:21:29.403549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403483    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tmp-dir\") pod \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\" (UID: \"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f\") "
Sep 19 06:21:29.403794 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403674    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-home" (OuterVolumeSpecName: "home") pod "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" (UID: "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.403794 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403703    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" (UID: "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.403881 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.403851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" (UID: "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.405838 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.405809    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" (UID: "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:21:29.405838 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.405820    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-dshm" (OuterVolumeSpecName: "dshm") pod "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" (UID: "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.405986 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.405882    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-kube-api-access-nx84n" (OuterVolumeSpecName: "kube-api-access-nx84n") pod "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" (UID: "f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f"). InnerVolumeSpecName "kube-api-access-nx84n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:21:29.504584 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.504548    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:29.504584 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.504577    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx84n\" (UniqueName: \"kubernetes.io/projected/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-kube-api-access-nx84n\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:29.504781 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.504610    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:29.504781 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.504618    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:29.504781 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.504633    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:29.504781 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.504641    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:29.632646 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.632566    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" containerID="a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720" exitCode=0
Sep 19 06:21:29.632646 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.632604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm" event={"ID":"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f","Type":"ContainerDied","Data":"a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720"}
Sep 19 06:21:29.632646 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.632631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm" event={"ID":"f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f","Type":"ContainerDied","Data":"cf060f3c25df56b80635b16abe34336506068a2dd3ab67b3b426467a5a84bd98"}
Sep 19 06:21:29.632646 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.632647    2581 scope.go:117] "RemoveContainer" containerID="a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720"
Sep 19 06:21:29.632915 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.632662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"
Sep 19 06:21:29.643573 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.643555    2581 scope.go:117] "RemoveContainer" containerID="a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720"
Sep 19 06:21:29.644081 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:21:29.644057    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720\": container with ID starting with a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720 not found: ID does not exist" containerID="a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720"
Sep 19 06:21:29.644190 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.644088    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720"} err="failed to get container status \"a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720\": rpc error: code = NotFound desc = could not find container \"a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720\": container with ID starting with a13f6dddad7540f1a000c71c96076b79c9d0c9759e88e99e8baaa5c86d145720 not found: ID does not exist"
Sep 19 06:21:29.646493 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.646469    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw" podUID="8bf08e71-b488-4aec-ba51-0eed41f8b422" containerName="main" containerID="cri-o://523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd" gracePeriod=45
Sep 19 06:21:29.654591 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.654570    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"]
Sep 19 06:21:29.660958 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.660937    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8f62b8ef-kserve-5db8997c9f-rfvjm"]
Sep 19 06:21:29.881426 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.881396    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:21:29.908355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-dshm\") pod \"8bf08e71-b488-4aec-ba51-0eed41f8b422\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") "
Sep 19 06:21:29.908355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-model-cache\") pod \"8bf08e71-b488-4aec-ba51-0eed41f8b422\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") "
Sep 19 06:21:29.908355 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908343    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf08e71-b488-4aec-ba51-0eed41f8b422-tls-certs\") pod \"8bf08e71-b488-4aec-ba51-0eed41f8b422\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") "
Sep 19 06:21:29.908638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptt6k\" (UniqueName: \"kubernetes.io/projected/8bf08e71-b488-4aec-ba51-0eed41f8b422-kube-api-access-ptt6k\") pod \"8bf08e71-b488-4aec-ba51-0eed41f8b422\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") "
Sep 19 06:21:29.908638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908396    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-tmp-dir\") pod \"8bf08e71-b488-4aec-ba51-0eed41f8b422\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") "
Sep 19 06:21:29.908638 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908474    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-home\") pod \"8bf08e71-b488-4aec-ba51-0eed41f8b422\" (UID: \"8bf08e71-b488-4aec-ba51-0eed41f8b422\") "
Sep 19 06:21:29.908817 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908713    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8bf08e71-b488-4aec-ba51-0eed41f8b422" (UID: "8bf08e71-b488-4aec-ba51-0eed41f8b422"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.908817 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908740    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-model-cache" (OuterVolumeSpecName: "model-cache") pod "8bf08e71-b488-4aec-ba51-0eed41f8b422" (UID: "8bf08e71-b488-4aec-ba51-0eed41f8b422"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.908934 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.908883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-home" (OuterVolumeSpecName: "home") pod "8bf08e71-b488-4aec-ba51-0eed41f8b422" (UID: "8bf08e71-b488-4aec-ba51-0eed41f8b422"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.910768 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.910738    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf08e71-b488-4aec-ba51-0eed41f8b422-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bf08e71-b488-4aec-ba51-0eed41f8b422" (UID: "8bf08e71-b488-4aec-ba51-0eed41f8b422"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:21:29.910768 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.910756    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-dshm" (OuterVolumeSpecName: "dshm") pod "8bf08e71-b488-4aec-ba51-0eed41f8b422" (UID: "8bf08e71-b488-4aec-ba51-0eed41f8b422"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:21:29.911058 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:29.911020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf08e71-b488-4aec-ba51-0eed41f8b422-kube-api-access-ptt6k" (OuterVolumeSpecName: "kube-api-access-ptt6k") pod "8bf08e71-b488-4aec-ba51-0eed41f8b422" (UID: "8bf08e71-b488-4aec-ba51-0eed41f8b422"). InnerVolumeSpecName "kube-api-access-ptt6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:21:30.009201 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.009169    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-dshm\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:30.009201 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.009195    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-model-cache\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:30.009201 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.009205    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf08e71-b488-4aec-ba51-0eed41f8b422-tls-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:30.009420 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.009214    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptt6k\" (UniqueName: \"kubernetes.io/projected/8bf08e71-b488-4aec-ba51-0eed41f8b422-kube-api-access-ptt6k\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:30.009420 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.009224    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-tmp-dir\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:30.009420 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.009232    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bf08e71-b488-4aec-ba51-0eed41f8b422-home\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:30.477653 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.477608    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" path="/var/lib/kubelet/pods/f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f/volumes"
Sep 19 06:21:30.639875 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.639840    2581 generic.go:356] "Generic (PLEG): container finished" podID="8bf08e71-b488-4aec-ba51-0eed41f8b422" containerID="523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd" exitCode=0
Sep 19 06:21:30.640081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.639880    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"
Sep 19 06:21:30.640081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.639925    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw" event={"ID":"8bf08e71-b488-4aec-ba51-0eed41f8b422","Type":"ContainerDied","Data":"523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd"}
Sep 19 06:21:30.640081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.639975    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw" event={"ID":"8bf08e71-b488-4aec-ba51-0eed41f8b422","Type":"ContainerDied","Data":"45aa86eaafeddaa6a22cc1a2e35f0ad56470fdbc997a1a6b9e3ba3573898d824"}
Sep 19 06:21:30.640081 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.639993    2581 scope.go:117] "RemoveContainer" containerID="523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd"
Sep 19 06:21:30.648834 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.648815    2581 scope.go:117] "RemoveContainer" containerID="523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd"
Sep 19 06:21:30.649148 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:21:30.649129    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd\": container with ID starting with 523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd not found: ID does not exist" containerID="523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd"
Sep 19 06:21:30.649257 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.649153    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd"} err="failed to get container status \"523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd\": rpc error: code = NotFound desc = could not find container \"523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd\": container with ID starting with 523ac182b3f093da04111b177b75d3445b1f6902cb774c25ed2b1b52c6535ffd not found: ID does not exist"
Sep 19 06:21:30.654996 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.654975    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"]
Sep 19 06:21:30.658508 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:30.658482    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-8f62b8ef-kserve-769858877f-xvtjw"]
Sep 19 06:21:32.477580 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:32.477545    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf08e71-b488-4aec-ba51-0eed41f8b422" path="/var/lib/kubelet/pods/8bf08e71-b488-4aec-ba51-0eed41f8b422/volumes"
Sep 19 06:21:33.649100 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:33.649065    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:21:34.071433 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071397    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"]
Sep 19 06:21:34.071757 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071745    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" containerName="main"
Sep 19 06:21:34.071810 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071760    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" containerName="main"
Sep 19 06:21:34.071810 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071773    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf08e71-b488-4aec-ba51-0eed41f8b422" containerName="main"
Sep 19 06:21:34.071810 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071779    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf08e71-b488-4aec-ba51-0eed41f8b422" containerName="main"
Sep 19 06:21:34.071908 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071834    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf08e71-b488-4aec-ba51-0eed41f8b422" containerName="main"
Sep 19 06:21:34.071908 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.071843    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e00cd2-cef1-4e6d-89d1-8fc32a410b7f" containerName="main"
Sep 19 06:21:34.076060 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.076020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.078548 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.078519    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 06:21:34.078750 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.078589    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nkngb\""
Sep 19 06:21:34.078855 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.078838    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 06:21:34.078905 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.078844    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 06:21:34.078954 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.078933    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 06:21:34.084151 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.084126    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"]
Sep 19 06:21:34.146348 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.146305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-metrics-certs\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.146543 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.146366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnmd\" (UniqueName: \"kubernetes.io/projected/1f4ed793-f1bc-47db-a6f8-d6925c12a225-kube-api-access-rmnmd\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.146543 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.146427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-maas-api-tls\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.146543 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.146482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ed793-f1bc-47db-a6f8-d6925c12a225-openshift-service-ca\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.247871 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.247834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ed793-f1bc-47db-a6f8-d6925c12a225-openshift-service-ca\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.248121 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.247896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-metrics-certs\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.248121 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.247928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnmd\" (UniqueName: \"kubernetes.io/projected/1f4ed793-f1bc-47db-a6f8-d6925c12a225-kube-api-access-rmnmd\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.248121 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.247960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-maas-api-tls\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.248549 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.248523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ed793-f1bc-47db-a6f8-d6925c12a225-openshift-service-ca\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.250596 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.250573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-maas-api-tls\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.250696 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.250615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-metrics-certs\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.255220 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.255198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnmd\" (UniqueName: \"kubernetes.io/projected/1f4ed793-f1bc-47db-a6f8-d6925c12a225-kube-api-access-rmnmd\") pod \"maas-api-5598d8c9fb-6lgkj\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.387593 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.387510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:34.657168 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.657141    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"]
Sep 19 06:21:34.658822 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:21:34.658796    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4ed793_f1bc_47db_a6f8_d6925c12a225.slice/crio-9f7625579ee27b18fd0e2e317386c754056da213a26767c8b18fe56cdeb91511 WatchSource:0}: Error finding container 9f7625579ee27b18fd0e2e317386c754056da213a26767c8b18fe56cdeb91511: Status 404 returned error can't find the container with id 9f7625579ee27b18fd0e2e317386c754056da213a26767c8b18fe56cdeb91511
Sep 19 06:21:34.660145 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:34.660120    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 06:21:35.667902 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:35.667864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj" event={"ID":"1f4ed793-f1bc-47db-a6f8-d6925c12a225","Type":"ContainerStarted","Data":"b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa"}
Sep 19 06:21:35.668376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:35.667926    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj" event={"ID":"1f4ed793-f1bc-47db-a6f8-d6925c12a225","Type":"ContainerStarted","Data":"9f7625579ee27b18fd0e2e317386c754056da213a26767c8b18fe56cdeb91511"}
Sep 19 06:21:35.668376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:35.668074    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:35.685988 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:35.685937    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj" podStartSLOduration=1.234569064 podStartE2EDuration="1.685923692s" totalImagesPullingTime="451.354628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:21:35.683987399 +0000 UTC m=+2441.294667381" watchObservedRunningTime="2026-09-19 06:21:35.685923692 +0000 UTC m=+2441.296603655"
Sep 19 06:21:41.241790 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.241709    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"]
Sep 19 06:21:41.242290 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.242018    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj" podUID="1f4ed793-f1bc-47db-a6f8-d6925c12a225" containerName="maas-api" containerID="cri-o://b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa" gracePeriod=30
Sep 19 06:21:41.247663 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.247637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:41.269756 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.269727    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"]
Sep 19 06:21:41.272947 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.272929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.283751 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.283727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"]
Sep 19 06:21:41.311993 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.311948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d32390-bc74-4ea1-9327-2f5bd626ac3f-openshift-service-ca\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.312187 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.312024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d32390-bc74-4ea1-9327-2f5bd626ac3f-maas-api-tls\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.312187 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.312108    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxd9\" (UniqueName: \"kubernetes.io/projected/59d32390-bc74-4ea1-9327-2f5bd626ac3f-kube-api-access-flxd9\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.312302 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.312187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d32390-bc74-4ea1-9327-2f5bd626ac3f-metrics-certs\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.413698 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.413652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d32390-bc74-4ea1-9327-2f5bd626ac3f-openshift-service-ca\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.413861 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.413714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d32390-bc74-4ea1-9327-2f5bd626ac3f-maas-api-tls\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.413861 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.413760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flxd9\" (UniqueName: \"kubernetes.io/projected/59d32390-bc74-4ea1-9327-2f5bd626ac3f-kube-api-access-flxd9\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.413861 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.413825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d32390-bc74-4ea1-9327-2f5bd626ac3f-metrics-certs\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.414992 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.414935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d32390-bc74-4ea1-9327-2f5bd626ac3f-openshift-service-ca\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.417171 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.417142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d32390-bc74-4ea1-9327-2f5bd626ac3f-maas-api-tls\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.417316 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.417222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d32390-bc74-4ea1-9327-2f5bd626ac3f-metrics-certs\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.422347 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.422323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxd9\" (UniqueName: \"kubernetes.io/projected/59d32390-bc74-4ea1-9327-2f5bd626ac3f-kube-api-access-flxd9\") pod \"maas-api-68c86dfd56-bg26v\" (UID: \"59d32390-bc74-4ea1-9327-2f5bd626ac3f\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.484693 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.484657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:41.515195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.515109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmnmd\" (UniqueName: \"kubernetes.io/projected/1f4ed793-f1bc-47db-a6f8-d6925c12a225-kube-api-access-rmnmd\") pod \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") "
Sep 19 06:21:41.515195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.515179    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-metrics-certs\") pod \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") "
Sep 19 06:21:41.515195 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.515201    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-maas-api-tls\") pod \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") "
Sep 19 06:21:41.515462 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.515266    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ed793-f1bc-47db-a6f8-d6925c12a225-openshift-service-ca\") pod \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\" (UID: \"1f4ed793-f1bc-47db-a6f8-d6925c12a225\") "
Sep 19 06:21:41.515716 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.515683    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4ed793-f1bc-47db-a6f8-d6925c12a225-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f4ed793-f1bc-47db-a6f8-d6925c12a225" (UID: "1f4ed793-f1bc-47db-a6f8-d6925c12a225"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:21:41.517509 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.517485    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f4ed793-f1bc-47db-a6f8-d6925c12a225" (UID: "1f4ed793-f1bc-47db-a6f8-d6925c12a225"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:21:41.517611 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.517551    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4ed793-f1bc-47db-a6f8-d6925c12a225-kube-api-access-rmnmd" (OuterVolumeSpecName: "kube-api-access-rmnmd") pod "1f4ed793-f1bc-47db-a6f8-d6925c12a225" (UID: "1f4ed793-f1bc-47db-a6f8-d6925c12a225"). InnerVolumeSpecName "kube-api-access-rmnmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:21:41.517850 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.517834    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1f4ed793-f1bc-47db-a6f8-d6925c12a225" (UID: "1f4ed793-f1bc-47db-a6f8-d6925c12a225"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:21:41.616778 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.616746    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmnmd\" (UniqueName: \"kubernetes.io/projected/1f4ed793-f1bc-47db-a6f8-d6925c12a225-kube-api-access-rmnmd\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:41.616778 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.616777    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-metrics-certs\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:41.616778 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.616785    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4ed793-f1bc-47db-a6f8-d6925c12a225-maas-api-tls\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:41.617010 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.616793    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ed793-f1bc-47db-a6f8-d6925c12a225-openshift-service-ca\") on node \"ip-10-0-131-202.ec2.internal\" DevicePath \"\""
Sep 19 06:21:41.628700 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.628656    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:41.701289 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.701252    2581 generic.go:356] "Generic (PLEG): container finished" podID="1f4ed793-f1bc-47db-a6f8-d6925c12a225" containerID="b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa" exitCode=0
Sep 19 06:21:41.701435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.701347    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj" event={"ID":"1f4ed793-f1bc-47db-a6f8-d6925c12a225","Type":"ContainerDied","Data":"b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa"}
Sep 19 06:21:41.701435 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.701398    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj" event={"ID":"1f4ed793-f1bc-47db-a6f8-d6925c12a225","Type":"ContainerDied","Data":"9f7625579ee27b18fd0e2e317386c754056da213a26767c8b18fe56cdeb91511"}
Sep 19 06:21:41.701515 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.701427    2581 scope.go:117] "RemoveContainer" containerID="b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa"
Sep 19 06:21:41.701619 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.701589    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"
Sep 19 06:21:41.713306 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.713284    2581 scope.go:117] "RemoveContainer" containerID="b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa"
Sep 19 06:21:41.713634 ip-10-0-131-202 kubenswrapper[2581]: E0919 06:21:41.713613    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa\": container with ID starting with b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa not found: ID does not exist" containerID="b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa"
Sep 19 06:21:41.713702 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.713641    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa"} err="failed to get container status \"b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa\": rpc error: code = NotFound desc = could not find container \"b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa\": container with ID starting with b6ca5721902973efc0acb402b90805907e1a8f810f8ab6f70b2a9f64003235fa not found: ID does not exist"
Sep 19 06:21:41.726166 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.726138    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"]
Sep 19 06:21:41.731224 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.731199    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6lgkj"]
Sep 19 06:21:41.744609 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.744577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:21:41.898635 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:41.898608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"]
Sep 19 06:21:41.900430 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:21:41.900401    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d32390_bc74_4ea1_9327_2f5bd626ac3f.slice/crio-a86163267e51acdf91707e24133fe51f010dbfa941b7e3458e1921f05c7fae57 WatchSource:0}: Error finding container a86163267e51acdf91707e24133fe51f010dbfa941b7e3458e1921f05c7fae57: Status 404 returned error can't find the container with id a86163267e51acdf91707e24133fe51f010dbfa941b7e3458e1921f05c7fae57
Sep 19 06:21:42.477621 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:42.477545    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f4ed793-f1bc-47db-a6f8-d6925c12a225" path="/var/lib/kubelet/pods/1f4ed793-f1bc-47db-a6f8-d6925c12a225/volumes"
Sep 19 06:21:42.707968 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:42.707907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v" event={"ID":"59d32390-bc74-4ea1-9327-2f5bd626ac3f","Type":"ContainerStarted","Data":"2adb96976dfa020c17eeda8a59b82e3fb09365c3b83d64348480df11c4b80cdd"}
Sep 19 06:21:42.708182 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:42.707982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v" event={"ID":"59d32390-bc74-4ea1-9327-2f5bd626ac3f","Type":"ContainerStarted","Data":"a86163267e51acdf91707e24133fe51f010dbfa941b7e3458e1921f05c7fae57"}
Sep 19 06:21:42.708182 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:42.708015    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:42.726164 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:42.726110    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v" podStartSLOduration=1.429577386 podStartE2EDuration="1.726096602s" totalImagesPullingTime="296.519216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:21:42.724338025 +0000 UTC m=+2448.335018001" watchObservedRunningTime="2026-09-19 06:21:42.726096602 +0000 UTC m=+2448.336776593"
Sep 19 06:21:47.842307 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:47.842260    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:21:48.719917 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:48.719887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-bg26v"
Sep 19 06:21:53.848442 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:53.848405    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:21:56.879459 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:21:56.879423    2581 scope.go:117] "RemoveContainer" containerID="0b58233179c470d8b47d061e80070db3d5479aa66e59ff56d65ad41b5a07a872"
Sep 19 06:22:01.540243 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:01.540206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-clnxg"]
Sep 19 06:22:09.208126 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:09.208088    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-p4q72_1077e5d4-7ffe-401e-a3f4-bf3ae4916663/manager/0.log"
Sep 19 06:22:09.331073 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:09.331022    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-px6dd_14dc59d9-81f4-44d4-81d3-d4cf52cb08fc/manager/0.log"
Sep 19 06:22:10.062806 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:10.062780    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-wsknt_9e36ec9c-d443-4b99-ad4b-dd31ee998567/copy-manifests/0.log"
Sep 19 06:22:10.081309 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:10.081281    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-wsknt_9e36ec9c-d443-4b99-ad4b-dd31ee998567/manager/0.log"
Sep 19 06:22:10.417636 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:10.417543    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68c86dfd56-bg26v_59d32390-bc74-4ea1-9327-2f5bd626ac3f/maas-api/0.log"
Sep 19 06:22:10.753512 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:10.753419    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29829975-sw5b5_1453eba6-e084-409e-b6a9-aa5174f2cff2/cleanup/0.log"
Sep 19 06:22:11.774463 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:11.774431    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q2v9j_e229b570-bffc-4161-ae3b-161067cde1ef/manager/0.log"
Sep 19 06:22:11.893168 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:11.893137    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hr2fp_3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4/manager/0.log"
Sep 19 06:22:12.011449 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:12.011426    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-wh695_f2592f39-d2fe-4de3-85d7-8d6fed48b5fb/kuadrant-console-plugin/0.log"
Sep 19 06:22:12.145239 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:12.145193    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-2bzdd_33c1058d-19a6-4181-9a8d-534d5eba7d7d/manager/0.log"
Sep 19 06:22:12.263417 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:12.263383    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-clnxg_9b8e91fb-1c4e-4158-a43a-dec85878969d/limitador/0.log"
Sep 19 06:22:12.388314 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:12.388282    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jjtpw_d9826c18-e90d-48b2-b98b-ea68840ffcc8/manager/0.log"
Sep 19 06:22:12.985199 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:12.985168    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-nfxsj_6ff8af0d-3a8c-43b7-887c-41185725a0b4/kube-auth-proxy/0.log"
Sep 19 06:22:13.583053 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:13.583007    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8667f66c84-kv9d7_a4c18157-81fa-45a4-b112-7beaed10c0ef/router/0.log"
Sep 19 06:22:14.164123 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:14.164091    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-qtd9v_9c2a86e9-46b0-430e-a0ef-56513d577b7b/main/0.log"
Sep 19 06:22:14.396005 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:14.395973    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhzl8d_9c8f9b06-5246-4c65-81ea-ce3e3d9091a8/main/0.log"
Sep 19 06:22:14.634182 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:14.634153    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-t58z2_ff877d56-3140-4233-bd2a-79f82f7b8f0e/main/0.log"
Sep 19 06:22:23.131043 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:23.131001    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hq78s_78f960ee-50d2-40db-a972-81da4ef0d346/global-pull-secret-syncer/0.log"
Sep 19 06:22:23.280796 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:23.280764    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8gb7c_e349d24e-28fc-4824-adad-292ac8dee6e9/konnectivity-agent/0.log"
Sep 19 06:22:23.300588 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:23.300554    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-202.ec2.internal_c990e3bd2b767d7e89440c7596b76213/haproxy/0.log"
Sep 19 06:22:28.274517 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:28.274430    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q2v9j_e229b570-bffc-4161-ae3b-161067cde1ef/manager/0.log"
Sep 19 06:22:28.296565 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:28.296541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hr2fp_3eaa1d0a-eaef-4bd3-8a40-3d8db9e651b4/manager/0.log"
Sep 19 06:22:28.318148 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:28.318129    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-wh695_f2592f39-d2fe-4de3-85d7-8d6fed48b5fb/kuadrant-console-plugin/0.log"
Sep 19 06:22:28.359666 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:28.359643    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-2bzdd_33c1058d-19a6-4181-9a8d-534d5eba7d7d/manager/0.log"
Sep 19 06:22:28.407240 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:28.407214    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-clnxg_9b8e91fb-1c4e-4158-a43a-dec85878969d/limitador/0.log"
Sep 19 06:22:28.489080 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:28.489020    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-jjtpw_d9826c18-e90d-48b2-b98b-ea68840ffcc8/manager/0.log"
Sep 19 06:22:30.325578 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:30.325545    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zz8np_491e7ff6-0304-445d-af8a-fab61e96da6c/node-exporter/0.log"
Sep 19 06:22:30.349145 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:30.349126    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zz8np_491e7ff6-0304-445d-af8a-fab61e96da6c/kube-rbac-proxy/0.log"
Sep 19 06:22:30.370408 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:30.370334    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zz8np_491e7ff6-0304-445d-af8a-fab61e96da6c/init-textfile/0.log"
Sep 19 06:22:33.019326 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.019297    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/1.log"
Sep 19 06:22:33.028111 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.028085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-64wfl_f9ee5577-b709-488c-a1da-75108799e433/console-operator/2.log"
Sep 19 06:22:33.688009 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.687977    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"]
Sep 19 06:22:33.688348 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.688334    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4ed793-f1bc-47db-a6f8-d6925c12a225" containerName="maas-api"
Sep 19 06:22:33.688397 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.688350    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4ed793-f1bc-47db-a6f8-d6925c12a225" containerName="maas-api"
Sep 19 06:22:33.688439 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.688405    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f4ed793-f1bc-47db-a6f8-d6925c12a225" containerName="maas-api"
Sep 19 06:22:33.691386 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.691367    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.693999 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.693975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5n4fs\"/\"default-dockercfg-96wgx\""
Sep 19 06:22:33.694119 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.693978    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5n4fs\"/\"kube-root-ca.crt\""
Sep 19 06:22:33.695270 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.695250    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5n4fs\"/\"openshift-service-ca.crt\""
Sep 19 06:22:33.699259 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.699238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"]
Sep 19 06:22:33.757708 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.757678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l27wq\" (UniqueName: \"kubernetes.io/projected/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-kube-api-access-l27wq\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.757859 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.757717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-lib-modules\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.757859 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.757740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-proc\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.757859 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.757818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-sys\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.757859 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.757838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-podres\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859203 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-sys\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-podres\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l27wq\" (UniqueName: \"kubernetes.io/projected/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-kube-api-access-l27wq\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-sys\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-podres\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859376 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-lib-modules\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859579 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-proc\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859579 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-lib-modules\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.859579 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.859492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-proc\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:33.867416 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:33.867396    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l27wq\" (UniqueName: \"kubernetes.io/projected/7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07-kube-api-access-l27wq\") pod \"perf-node-gather-daemonset-7gjc6\" (UID: \"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:34.002437 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.002359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:34.271445 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.271416    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"]
Sep 19 06:22:34.274118 ip-10-0-131-202 kubenswrapper[2581]: W0919 06:22:34.274088    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7fa6cc33_3bd0_45b9_8ebf_02d1230d1c07.slice/crio-cb2bbee7b7572b65efe77e2c8957dfc6b8f1b41d94be1a91af53cbfc6cdd6695 WatchSource:0}: Error finding container cb2bbee7b7572b65efe77e2c8957dfc6b8f1b41d94be1a91af53cbfc6cdd6695: Status 404 returned error can't find the container with id cb2bbee7b7572b65efe77e2c8957dfc6b8f1b41d94be1a91af53cbfc6cdd6695
Sep 19 06:22:34.283635 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.283610    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-bnp4j_1c599698-b28e-4617-bbb1-45c8cfae0673/volume-data-source-validator/0.log"
Sep 19 06:22:34.961254 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.961219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6" event={"ID":"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07","Type":"ContainerStarted","Data":"d448254b4dcee12434b69589a2eaac20a78e26933cdc73632fc3cd7da111793d"}
Sep 19 06:22:34.961433 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.961275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6" event={"ID":"7fa6cc33-3bd0-45b9-8ebf-02d1230d1c07","Type":"ContainerStarted","Data":"cb2bbee7b7572b65efe77e2c8957dfc6b8f1b41d94be1a91af53cbfc6cdd6695"}
Sep 19 06:22:34.961433 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.961307    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:34.976304 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:34.976264    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6" podStartSLOduration=1.9762512220000001 podStartE2EDuration="1.976251222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 06:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:22:34.97569035 +0000 UTC m=+2500.586370329" watchObservedRunningTime="2026-09-19 06:22:34.976251222 +0000 UTC m=+2500.586931186"
Sep 19 06:22:35.457330 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:35.457303    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-htq54_6336f700-eea4-4f3a-ba95-a96d1167afc2/dns/0.log"
Sep 19 06:22:35.475379 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:35.475352    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-htq54_6336f700-eea4-4f3a-ba95-a96d1167afc2/kube-rbac-proxy/0.log"
Sep 19 06:22:35.594782 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:35.594752    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pcmxb_da9d45fc-33f1-4fad-b8ba-35bedabba559/dns-node-resolver/0.log"
Sep 19 06:22:36.126248 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:36.126215    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79657c785d-ljnjl_e2e80818-0d54-48a3-ba42-52d569f51fee/registry/0.log"
Sep 19 06:22:36.169886 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:36.169864    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wrrj9_b39f1022-4cf2-4513-a0fb-92216b5a67e3/node-ca/0.log"
Sep 19 06:22:37.593446 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:37.593370    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-nfxsj_6ff8af0d-3a8c-43b7-887c-41185725a0b4/kube-auth-proxy/0.log"
Sep 19 06:22:37.736170 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:37.736139    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8667f66c84-kv9d7_a4c18157-81fa-45a4-b112-7beaed10c0ef/router/0.log"
Sep 19 06:22:38.675019 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:38.674988    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-79scr_ebde495e-633a-4484-b386-4d650b0011ca/serve-healthcheck-canary/0.log"
Sep 19 06:22:39.294332 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:39.294281    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8lb8_53392f56-cf45-4b07-8cfc-fc8d56369ff9/kube-rbac-proxy/0.log"
Sep 19 06:22:39.312142 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:39.312122    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8lb8_53392f56-cf45-4b07-8cfc-fc8d56369ff9/exporter/0.log"
Sep 19 06:22:39.331564 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:39.331543    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p8lb8_53392f56-cf45-4b07-8cfc-fc8d56369ff9/extractor/0.log"
Sep 19 06:22:40.976110 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:40.976077    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-7gjc6"
Sep 19 06:22:41.501791 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:41.501762    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-p4q72_1077e5d4-7ffe-401e-a3f4-bf3ae4916663/manager/0.log"
Sep 19 06:22:41.936297 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:41.936264    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-px6dd_14dc59d9-81f4-44d4-81d3-d4cf52cb08fc/manager/0.log"
Sep 19 06:22:42.330480 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:42.330447    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-wsknt_9e36ec9c-d443-4b99-ad4b-dd31ee998567/manager/0.log"
Sep 19 06:22:42.347207 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:42.347185    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-wsknt_9e36ec9c-d443-4b99-ad4b-dd31ee998567/copy-manifests/0.log"
Sep 19 06:22:44.614388 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:44.614360    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-nfxsj_6ff8af0d-3a8c-43b7-887c-41185725a0b4/kube-auth-proxy/0.log"
Sep 19 06:22:44.729971 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:44.729894    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8667f66c84-kv9d7_a4c18157-81fa-45a4-b112-7beaed10c0ef/router/0.log"
Sep 19 06:22:46.662291 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:46.662258    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-nfxsj_6ff8af0d-3a8c-43b7-887c-41185725a0b4/kube-auth-proxy/0.log"
Sep 19 06:22:46.772795 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:46.772744    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8667f66c84-kv9d7_a4c18157-81fa-45a4-b112-7beaed10c0ef/router/0.log"
Sep 19 06:22:49.351243 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:49.351203    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-svbh9_f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3/kube-storage-version-migrator-operator/1.log"
Sep 19 06:22:49.352957 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:49.352931    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-svbh9_f0e6ed25-f8e9-4d1e-ab7b-f5d18a2483e3/kube-storage-version-migrator-operator/0.log"
Sep 19 06:22:50.626972 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.626946    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/kube-multus-additional-cni-plugins/0.log"
Sep 19 06:22:50.647992 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.647959    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/egress-router-binary-copy/0.log"
Sep 19 06:22:50.668381 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.668358    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/cni-plugins/0.log"
Sep 19 06:22:50.686241 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.686216    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/bond-cni-plugin/0.log"
Sep 19 06:22:50.704700 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.704673    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/routeoverride-cni/0.log"
Sep 19 06:22:50.722430 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.722405    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/whereabouts-cni-bincopy/0.log"
Sep 19 06:22:50.741567 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.741541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xb7qg_2b530891-1fbd-4286-a117-e437388721eb/whereabouts-cni/0.log"
Sep 19 06:22:50.801532 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.801504    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qp5nd_98470b56-3241-4618-9bf4-d6578d14c33d/kube-multus/0.log"
Sep 19 06:22:50.898089 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.898006    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cnxxg_6c42040f-8e4f-4d40-b199-58def6a92b99/network-metrics-daemon/0.log"
Sep 19 06:22:50.911797 ip-10-0-131-202 kubenswrapper[2581]: I0919 06:22:50.911763    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cnxxg_6c42040f-8e4f-4d40-b199-58def6a92b99/kube-rbac-proxy/0.log"