Sep 14 07:46:59.124971 ip-10-0-131-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 14 07:46:59.124983 ip-10-0-131-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 14 07:46:59.124992 ip-10-0-131-236 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 14 07:46:59.125174 ip-10-0-131-236 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 14 07:47:09.220207 ip-10-0-131-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 14 07:47:09.220226 ip-10-0-131-236 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ae853b3d64ee49978d6bacfaa9b4ae7d --
Sep 14 07:49:03.377255 ip-10-0-131-236 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 07:49:03.795869 ip-10-0-131-236 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:03.795869 ip-10-0-131-236 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 07:49:03.795869 ip-10-0-131-236 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:03.795869 ip-10-0-131-236 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 07:49:03.795869 ip-10-0-131-236 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:03.798894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.798809    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 07:49:03.803602 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803587    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:03.803602 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803601    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803605    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803608    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803611    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803613    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803616    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803619    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803622    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803628    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803631    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803634    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803637    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803639    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803642    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803644    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803647    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803651    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803653    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803656    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803659    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:03.803666 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803661    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803664    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803666    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803671    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803675    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803678    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803681    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803684    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803687    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803690    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803692    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803695    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803697    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803700    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803703    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803705    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803708    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803710    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803713    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:03.804159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803716    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803718    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803721    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803724    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803727    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803729    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803732    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803735    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803737    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803740    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803742    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803745    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803747    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803750    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803753    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803755    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803758    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803760    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803763    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803766    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:03.804628 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803768    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803772    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803775    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803778    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803780    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803783    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803785    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803788    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803791    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803793    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803795    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803798    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803800    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803803    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803805    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803808    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803813    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803816    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803819    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803822    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:03.805179 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803825    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803828    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803830    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803833    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803835    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803838    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803841    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803843    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803846    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803849    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803851    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803854    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803856    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803859    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803862    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803864    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803867    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803869    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:03.805677 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.803872    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:03.806341 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806328    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:03.806341 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806341    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806344    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806347    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806352    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806357    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806360    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806362    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806365    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806368    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806371    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806374    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806376    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806379    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806381    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806384    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806387    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806390    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806392    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806395    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806397    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:03.806396 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806400    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806403    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806406    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806409    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806411    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806414    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806417    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806419    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806422    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806424    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806427    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806429    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806432    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806434    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806437    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806439    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806442    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806444    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806446    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806449    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:03.806902 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806451    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806454    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806457    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806459    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806463    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806465    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806468    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806470    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806473    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806475    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806477    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806480    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806482    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806486    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806491    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806494    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806497    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806500    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806503    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:03.807410 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806506    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806508    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806511    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806513    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806516    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806518    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806521    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806523    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806526    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806528    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806531    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806533    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806536    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806538    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806541    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806544    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806547    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806549    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806552    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806555    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:03.807878 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806557    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806561    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806563    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806566    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806576    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806580    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806582    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806586    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806588    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806591    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806594    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806597    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806600    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806602    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806605    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806607    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806610    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806614    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.806617    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:03.808384 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807014    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807020    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807024    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807027    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807030    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807033    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807036    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807039    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807042    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807044    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807047    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807050    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807053    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807056    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807058    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807061    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807063    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807066    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807069    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807071    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:03.808867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807075    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807078    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807081    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807083    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807086    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807088    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807091    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807094    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807097    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807099    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807102    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807105    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807107    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807110    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807113    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807115    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807118    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807120    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807123    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807125    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:03.809368 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807128    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807131    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807134    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807136    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807139    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807142    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807145    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807148    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807150    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807153    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807155    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807158    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807161    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807164    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807167    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807169    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807172    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807174    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807177    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807180    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:03.809858 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807184    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807187    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807190    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807193    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807196    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807200    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807203    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807206    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807208    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807211    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807214    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807217    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807219    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807222    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807225    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807227    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807230    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807232    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807236    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:03.810404 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807238    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807241    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807243    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807246    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807251    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807253    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807256    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807259    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807262    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807264    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807267    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807269    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807272    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807274    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807277    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807279    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807282    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807284    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807288    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.807292    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:03.810867 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807363    2556 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807400    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807406    2556 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807410    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807414    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807418    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807422    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807426    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807430    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807433    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807436    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807439    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807442    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807445    2556 flags.go:64] FLAG: --cgroup-root=""
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807448    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807451    2556 flags.go:64] FLAG: --client-ca-file=""
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807455    2556 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807458    2556 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807462    2556 flags.go:64] FLAG: --cluster-domain=""
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807464    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807468    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807471    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807474    2556 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807478    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 07:49:03.811382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807480    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807483    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807487    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807490    2556 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807493    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807496    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807499    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807502    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807506    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807509    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807512    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807514    2556 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807517    2556 flags.go:64] FLAG: --enable-server="true"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807520    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807524    2556 flags.go:64] FLAG: --event-burst="100"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807527    2556 flags.go:64] FLAG: --event-qps="50"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807530    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807533    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807536    2556 flags.go:64] FLAG: --eviction-hard=""
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807539    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807542    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807545    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807548    2556 flags.go:64] FLAG: --eviction-soft=""
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807551    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807554    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 07:49:03.811956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807557    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807560    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807563    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807566    2556 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807569    2556 flags.go:64] FLAG: --feature-gates=""
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807573    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807576    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807579    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807582    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807585    2556 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807588    2556 flags.go:64] FLAG: --help="false"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807592    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-131-236.ec2.internal"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807594    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807597    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807600    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807604    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807607    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807610    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807612    2556 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807615    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807618    2556 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807621    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807624    2556 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807627    2556 flags.go:64] FLAG: --kube-reserved=""
Sep 14 07:49:03.812604 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807632    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807635    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807638    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807641    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807644    2556 flags.go:64] FLAG: --lock-file=""
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807646    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807649    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807652    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807657    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807661    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807663    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807666    2556 flags.go:64] FLAG: --logging-format="text"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807669    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807672    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807675    2556 flags.go:64] FLAG: --manifest-url=""
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807678    2556 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807682    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807686    2556 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807689    2556 flags.go:64] FLAG: --max-pods="110"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807692    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807695    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807698    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807701    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807704    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807706    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 07:49:03.813202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807710    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807716    2556 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807719    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807722    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807725    2556 flags.go:64] FLAG: --pod-cidr=""
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807728    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807732    2556 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807737    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807740    2556 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807744    2556 flags.go:64] FLAG: --port="10250"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807747    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807750    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01c62c781208730e7"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807753    2556 flags.go:64] FLAG: --qos-reserved=""
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807756    2556 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807759    2556 flags.go:64] FLAG: --register-node="true"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807762    2556 flags.go:64] FLAG: --register-with-taints=""
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807766    2556 flags.go:64] FLAG: --registry-burst="10"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807768    2556 flags.go:64] FLAG: --registry-qps="5"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807771    2556 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807774    2556 flags.go:64] FLAG: --reserved-memory=""
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807778    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807781    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807783    2556 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807786    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807789    2556 flags.go:64] FLAG: --runonce="false"
Sep 14 07:49:03.813837 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807792    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807795    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807798    2556 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807801    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807804    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807807    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807810    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807813    2556 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807816    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807819    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807822    2556 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807824    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807827    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807830    2556 flags.go:64] FLAG: --system-cgroups=""
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807834    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807839    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807842    2556 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807845    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807849    2556 flags.go:64] FLAG: --tls-min-version=""
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807851    2556 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807854    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807859    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807862    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807865    2556 flags.go:64] FLAG: --v="2"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807869    2556 flags.go:64] FLAG: --version="false"
Sep 14 07:49:03.814567 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807872    2556 flags.go:64] FLAG: --vmodule=""
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807876    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.807880    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808006    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808010    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808013    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808016    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808020    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808024    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808027    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808031    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808034    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808036    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808039    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808042    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808045    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808048    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808051    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808054    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808056    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:03.815597 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808059    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808063    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808066    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808068    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808071    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808074    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808076    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808079    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808083    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808085    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808088    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808090    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808093    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808096    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808098    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808100    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808103    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808106    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808108    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808111    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:03.816113 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808113    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808116    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808119    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808121    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808124    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808126    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808129    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808131    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808134    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808136    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808139    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808141    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808144    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808147    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808150    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808153    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808155    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808158    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808161    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:03.816610 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808164    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808169    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808172    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808175    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808177    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808180    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808183    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808185    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808188    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808190    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808193    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808197    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808201    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808204    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808207    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808209    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808212    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808215    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808217    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:03.817092 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808220    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808222    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808225    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808228    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808230    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808233    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808235    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808239    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808241    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808246    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808249    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808252    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808254    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808257    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808261    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808265    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808267    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808270    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808273    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808276    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:03.817568 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808279    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808282    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808284    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.808287    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.808816    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.816990    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.817005    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.817855    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.817861    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817919    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817924    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817928    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817932    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817935    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:03.818201 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817939    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817943    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817946    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817949    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817952    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817956    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817972    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817977    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817981    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817984    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817988    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817993    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.817997    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818000    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818003    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818006    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818008    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818011    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818014    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818016    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:03.818556 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818026    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818030    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818032    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818035    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818038    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818040    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818043    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818046    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818048    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818051    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818053    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818056    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818060    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818063    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818066    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818069    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818072    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818074    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818078    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:03.819062 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818081    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818084    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818088    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818090    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818093    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818095    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818098    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818100    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818103    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818105    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818108    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818110    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818113    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818116    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818119    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818122    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818124    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818127    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818129    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818132    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:03.819530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818135    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818137    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818140    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818142    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818145    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818148    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818150    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818153    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818156    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818159    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818161    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818164    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818166    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818169    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818171    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818174    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818177    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818179    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818182    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818184    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:03.820042 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818187    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818189    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818191    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818194    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818197    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818199    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818202    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818205    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818207    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818210    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818213    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818216    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818218    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818221    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:03.820511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818223    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.818228    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818341    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818346    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818349    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818352    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818355    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818358    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818361    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818364    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818366    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818369    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818372    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818374    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818378    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:03.820896 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818381    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818383    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818386    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818389    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818391    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818394    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818396    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818398    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818401    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818404    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818407    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818409    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818412    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818415    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818417    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818420    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818423    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818425    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818428    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818430    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818433    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:03.821288 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818435    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818438    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818440    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818442    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818445    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818447    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818450    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818452    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818455    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818457    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818460    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818463    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818466    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818468    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818470    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818474    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818477    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818480    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818483    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818486    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:03.821802 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818490    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818492    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818495    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818498    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818501    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818504    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818506    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818509    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818511    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818514    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818516    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818518    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818521    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818523    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818526    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818528    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818531    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818533    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818536    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:03.822291 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818538    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818541    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818543    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818545    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818548    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818551    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818553    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818556    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818559    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818564    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818574    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818579    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818581    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818584    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818587    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818590    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818592    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818595    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818597    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:03.822754 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818600    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818603    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818605    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818608    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818610    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818613    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:03.818615    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.818620    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.818711    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.820777    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.822188    2556 server.go:1019] "Starting client certificate rotation"
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.822294    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:49:03.823264 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.822332    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:49:03.824915 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.824901    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:49:03.826811 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.826785    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:49:03.837337 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.837322    2556 log.go:25] "Validated CRI v1 runtime API"
Sep 14 07:49:03.842379 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.842362    2556 log.go:25] "Validated CRI v1 image API"
Sep 14 07:49:03.843731 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.843717    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 07:49:03.846804 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.846781    2556 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ca517414-bec8-4789-a2ad-b05b260c71f1:/dev/nvme0n1p3 e06ee198-ab55-4386-b31f-5d4bd97953a0:/dev/nvme0n1p4]
Sep 14 07:49:03.846863 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.846803    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 07:49:03.854379 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.854212    2556 manager.go:216] Machine: {Timestamp:2026-09-14 07:49:03.852575225 +0000 UTC m=+0.351968022 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099818 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208af14dbead2faf3830fedd53b1f2 SystemUUID:ec208af1-4dbe-ad2f-af38-30fedd53b1f2 BootID:ae853b3d-64ee-4997-8d6b-acfaa9b4ae7d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d6:2e:ba:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d6:2e:ba:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:3f:46:16:37:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 07:49:03.854379 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.854367    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 07:49:03.854721 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.854699    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 07:49:03.856469 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.856451    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:49:03.858045 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.858020    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 07:49:03.858176 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.858050    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 07:49:03.858840 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.858830    2556 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 07:49:03.858875 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.858842    2556 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 07:49:03.858875 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.858849    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:49:03.859616 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.859606    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:49:03.860617 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.860608    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 07:49:03.864322 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864310    2556 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:49:03.864436 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864426    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 07:49:03.864471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864455    2556 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 07:49:03.864471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864464    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 07:49:03.864530 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864478    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 07:49:03.864530 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864487    2556 kubelet.go:398] "Adding apiserver pod source"
Sep 14 07:49:03.864530 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864496    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 07:49:03.864898 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864886    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:49:03.864935 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.864909    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:49:03.867356 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.867337    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 07:49:03.869069 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.869055    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 07:49:03.869109 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.869086    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 07:49:03.871207 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871194    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 07:49:03.871256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871211    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 07:49:03.871256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871217    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 07:49:03.871256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871222    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 07:49:03.871256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871230    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 07:49:03.871256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871236    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 07:49:03.871256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871255    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 07:49:03.871403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871262    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 07:49:03.871403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871269    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 07:49:03.871403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871274    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 07:49:03.871403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871283    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 07:49:03.871403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.871288    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 07:49:03.873091 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.873080    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 07:49:03.873123 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.873092    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 07:49:03.876282 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.876247    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 07:49:03.876363 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.876305    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 07:49:03.876363 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.876316    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 07:49:03.876956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.876943    2556 server.go:1268] "Started kubelet"
Sep 14 07:49:03.877056 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.877023    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 07:49:03.877162 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.877085    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 07:49:03.877219 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.877172    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 07:49:03.877957 ip-10-0-131-236 systemd[1]: Started Kubernetes Kubelet.
Sep 14 07:49:03.878808 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.878786    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 07:49:03.879545 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.879509    2556 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 07:49:03.882467 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.882444    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 07:49:03.882545 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.881648    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-236.ec2.internal.18d5207bfec33926  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-236.ec2.internal,UID:ip-10-0-131-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-236.ec2.internal,},FirstTimestamp:2026-09-14 07:49:03.876913446 +0000 UTC m=+0.376306214,LastTimestamp:2026-09-14 07:49:03.876913446 +0000 UTC m=+0.376306214,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-236.ec2.internal,}"
Sep 14 07:49:03.883317 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.883296    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 07:49:03.884012 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.883999    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 07:49:03.885063 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.884795    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:03.885063 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.884842    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 07:49:03.885063 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.884976    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 07:49:03.885227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.885170    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 07:49:03.885227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.885214    2556 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 07:49:03.885227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.885222    2556 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 07:49:03.886117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886100    2556 factory.go:55] Registering systemd factory
Sep 14 07:49:03.886117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886116    2556 factory.go:223] Registration of the systemd container factory successfully
Sep 14 07:49:03.886354 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886338    2556 factory.go:153] Registering CRI-O factory
Sep 14 07:49:03.886426 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886358    2556 factory.go:223] Registration of the crio container factory successfully
Sep 14 07:49:03.886426 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886412    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 07:49:03.886525 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886431    2556 factory.go:103] Registering Raw factory
Sep 14 07:49:03.886525 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886448    2556 manager.go:1196] Started watching for new ooms in manager
Sep 14 07:49:03.886865 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.886854    2556 manager.go:349] Starting recovery of all containers
Sep 14 07:49:03.887260 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.887236    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 07:49:03.887377 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.887352    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 07:49:03.892195 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.892164    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 07:49:03.899183 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.899049    2556 manager.go:354] Recovery completed
Sep 14 07:49:03.903876 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.903861    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:03.906472 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.906451    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:03.906541 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.906488    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:03.906541 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.906504    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:03.907047 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907029    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 07:49:03.907047 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907045    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:03.907193 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907046    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 07:49:03.907193 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907097    2556 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:49:03.907193 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907174    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g866m"
Sep 14 07:49:03.907873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907856    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:03.907935 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907888    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:03.907935 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.907912    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:03.908566 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.908495    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-236.ec2.internal.18d5207c00863e87  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-236.ec2.internal,UID:ip-10-0-131-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-236.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-236.ec2.internal,},FirstTimestamp:2026-09-14 07:49:03.906471559 +0000 UTC m=+0.405864330,LastTimestamp:2026-09-14 07:49:03.906471559 +0000 UTC m=+0.405864330,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-236.ec2.internal,}"
Sep 14 07:49:03.909346 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.909333    2556 policy_none.go:49] "None policy: Start"
Sep 14 07:49:03.909400 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.909350    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 07:49:03.909400 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.909360    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 07:49:03.911305 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.911287    2556 policy_none.go:47] "Start"
Sep 14 07:49:03.913027 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.913011    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g866m"
Sep 14 07:49:03.918471 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.918373    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-236.ec2.internal.18d5207c00869e40  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-236.ec2.internal,UID:ip-10-0-131-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-236.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-236.ec2.internal,},FirstTimestamp:2026-09-14 07:49:03.906496064 +0000 UTC m=+0.405888834,LastTimestamp:2026-09-14 07:49:03.906496064 +0000 UTC m=+0.405888834,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-236.ec2.internal,}"
Sep 14 07:49:03.940783 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.940759    2556 manager.go:337] "Starting Device Plugin manager"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.940814    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.940829    2556 server.go:87] "Starting device plugin registration server"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.941168    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.941188    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.941284    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.941392    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:03.941413    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.942165    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 07:49:03.962873 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:03.942212    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.012118 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.012088    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 07:49:04.012227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.012142    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 07:49:04.012227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.012169    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 07:49:04.012312 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.012268    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 07:49:04.014601 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.014582    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:04.042704 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.042683    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.043680 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.043664    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.043767 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.043694    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.043767 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.043711    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.043767 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.043733    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.053448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.053381    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.053448 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.053402    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-236.ec2.internal\": node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.075133 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.075112    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.113131 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.113110    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"]
Sep 14 07:49:04.113214 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.113196    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.114495 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.114482    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.114558 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.114510    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.114558 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.114520    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.116814 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.116802    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.116953 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.116938    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.117015 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.116980    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.117427 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.117412    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.117513 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.117446    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.117513 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.117456    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.117513 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.117496    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.117654 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.117521    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.117654 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.117536    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.119683 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.119668    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.119759 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.119694    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.120347 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.120331    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.120440 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.120368    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.120440 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.120387    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.143510 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.143493    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-236.ec2.internal\" not found" node="ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.146862 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.146849    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-236.ec2.internal\" not found" node="ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.175685 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.175656    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.187223 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.187206    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e9041db37f31c68c35286bb7d3caee1d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"e9041db37f31c68c35286bb7d3caee1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.187277 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.187228    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9041db37f31c68c35286bb7d3caee1d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"e9041db37f31c68c35286bb7d3caee1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.187277 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.187245    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b56fcbc9aa34740ef36ad85c9970517-config\") pod \"kube-apiserver-proxy-ip-10-0-131-236.ec2.internal\" (UID: \"6b56fcbc9aa34740ef36ad85c9970517\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.276298 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.276275    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.287612 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.287587    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e9041db37f31c68c35286bb7d3caee1d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"e9041db37f31c68c35286bb7d3caee1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.287674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.287617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9041db37f31c68c35286bb7d3caee1d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"e9041db37f31c68c35286bb7d3caee1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.287674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.287632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b56fcbc9aa34740ef36ad85c9970517-config\") pod \"kube-apiserver-proxy-ip-10-0-131-236.ec2.internal\" (UID: \"6b56fcbc9aa34740ef36ad85c9970517\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.287674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.287657    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b56fcbc9aa34740ef36ad85c9970517-config\") pod \"kube-apiserver-proxy-ip-10-0-131-236.ec2.internal\" (UID: \"6b56fcbc9aa34740ef36ad85c9970517\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.287770 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.287692    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9041db37f31c68c35286bb7d3caee1d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"e9041db37f31c68c35286bb7d3caee1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.287770 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.287698    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e9041db37f31c68c35286bb7d3caee1d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"e9041db37f31c68c35286bb7d3caee1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.377051 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.376997    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.449278 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.449247    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.450408 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.450381    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:04.478065 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.478041    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.583566 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.583518    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.686144 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.686065    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.787197 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.787159    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.822196 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.822160    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 07:49:04.822695 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.822321    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:49:04.841042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.841021    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:04.888366 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.888339    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:04.918144 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.918099    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 07:44:04 +0000 UTC" deadline="2028-04-29 14:05:48.799335521 +0000 UTC"
Sep 14 07:49:04.918144 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:04.918139    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14238h16m43.881199869s"
Sep 14 07:49:04.989221 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:04.989196    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:05.089445 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.089423    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:05.101876 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.101854    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:05.102086 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:05.102060    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9041db37f31c68c35286bb7d3caee1d.slice/crio-7d42defcc0d67d831b4689251f9f1a0debae3368101d2b1bf66472c74b28d35a WatchSource:0}: Error finding container 7d42defcc0d67d831b4689251f9f1a0debae3368101d2b1bf66472c74b28d35a: Status 404 returned error can't find the container with id 7d42defcc0d67d831b4689251f9f1a0debae3368101d2b1bf66472c74b28d35a
Sep 14 07:49:05.102971 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.102953    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:05.103077 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.103004    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:05.103077 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.103016    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:05.107274 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.107258    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:49:05.166059 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.166031    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:05.189606 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.189579    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:05.257734 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:05.257705    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b56fcbc9aa34740ef36ad85c9970517.slice/crio-d8fd8c8a59e958dbebeab01979182c737dd1dc9dae408be24c90537410ccd3b0 WatchSource:0}: Error finding container d8fd8c8a59e958dbebeab01979182c737dd1dc9dae408be24c90537410ccd3b0: Status 404 returned error can't find the container with id d8fd8c8a59e958dbebeab01979182c737dd1dc9dae408be24c90537410ccd3b0
Sep 14 07:49:05.289974 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.289943    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:05.390217 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.390182    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Sep 14 07:49:05.405880 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.405858    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:05.485602 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.485565    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:05.495223 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.495193    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:49:05.496391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.496377    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Sep 14 07:49:05.508274 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.508221    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:49:05.866110 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.865807    2556 apiserver.go:52] "Watching apiserver"
Sep 14 07:49:05.873218 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.873155    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 07:49:05.873829 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.873800    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal","openshift-multus/multus-jn2x4","openshift-multus/network-metrics-daemon-k7dvq","openshift-dns/node-resolver-ptgdj","openshift-image-registry/node-ca-wpgcg","openshift-multus/multus-additional-cni-plugins-hm6hk","openshift-network-diagnostics/network-check-target-n5ks4","openshift-network-operator/iptables-alerter-glvzr","openshift-ovn-kubernetes/ovnkube-node-cth69","kube-system/global-pull-secret-syncer-tc6ts","kube-system/konnectivity-agent-wrck8","kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv","openshift-cluster-node-tuning-operator/tuned-slx9r"]
Sep 14 07:49:05.876686 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.876620    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:05.880135 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.879138    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fr5zv\""
Sep 14 07:49:05.880135 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.879382    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 07:49:05.880135 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.879567    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.881220 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.881017    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 07:49:05.882488 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.881593    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hf4cz\""
Sep 14 07:49:05.882488 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.881977    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.882488 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.882266    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.882678 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.882518    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:05.882678 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.882607    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:05.885565 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.883866    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 07:49:05.885565 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.884550    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.887714 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.887017    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:05.888827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.888307    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wgqx9\""
Sep 14 07:49:05.888827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.888497    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.888827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.888674    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.890306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.889138    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:05.890306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.889171    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.890306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.889482    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bxgjx\""
Sep 14 07:49:05.890306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.889692    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 07:49:05.890306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.889999    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.891724 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.891702    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.892056 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.892035    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.892159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.892112    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 07:49:05.892159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.892145    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wvkx4\""
Sep 14 07:49:05.892669 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.892426    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:05.892669 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.892498    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:05.894751 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.894730    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:05.895677 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895651    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b39851f-2475-4f3e-b814-a2dbd27f3787-tmp\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.895768 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895704    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:05.895768 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895755    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nq2m\" (UniqueName: \"kubernetes.io/projected/be3af394-a1c5-4410-be29-eff8e1a01aa5-kube-api-access-6nq2m\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:05.895856 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895785    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfqx5\" (UniqueName: \"kubernetes.io/projected/378da416-3160-4554-99c3-9f79d9b34aab-kube-api-access-wfqx5\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.895895 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895884    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/618ca54d-af1f-4f80-a9e9-e93710b99d5b-host\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895923    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/618ca54d-af1f-4f80-a9e9-e93710b99d5b-serviceca\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.895961    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05bfe619-feda-4437-a91f-4a26bcf42615-konnectivity-ca\") pod \"konnectivity-agent-wrck8\" (UID: \"05bfe619-feda-4437-a91f-4a26bcf42615\") " pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896011    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-run\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-var-lib-kubelet\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896174    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-host\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05bfe619-feda-4437-a91f-4a26bcf42615-agent-certs\") pod \"konnectivity-agent-wrck8\" (UID: \"05bfe619-feda-4437-a91f-4a26bcf42615\") " pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896231    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-kubernetes\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896260    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-systemd\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896307    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-tuned\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896337    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-modprobe-d\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.896532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896455    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysconfig\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.897042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896597    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/378da416-3160-4554-99c3-9f79d9b34aab-tmp-dir\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.897042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nrsj\" (UniqueName: \"kubernetes.io/projected/618ca54d-af1f-4f80-a9e9-e93710b99d5b-kube-api-access-6nrsj\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:05.897042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896699    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysctl-d\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.897042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896760    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysctl-conf\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.897042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwbpd\" (UniqueName: \"kubernetes.io/projected/4b39851f-2475-4f3e-b814-a2dbd27f3787-kube-api-access-rwbpd\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.897042 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.896852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/378da416-3160-4554-99c3-9f79d9b34aab-hosts-file\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.897306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.897256    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-sys\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.897343 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.897306    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-lib-modules\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.898016 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.897867    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.898016 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.897917    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 07:49:05.899009 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.898393    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:05.899009 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.898874    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.900342 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.899444    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p5vq8\""
Sep 14 07:49:05.903248 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.903194    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 07:49:05.903362 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.903253    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 07:49:05.903466 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.903408    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.903527 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.903515    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:05.903657 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.903588    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:05.904147 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.903971    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.904147 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.904000    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 07:49:05.906971 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.905328    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 07:49:05.906971 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.905579    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k96cq\""
Sep 14 07:49:05.906971 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.906609    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:05.908978 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.908956    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:05.909563 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.909402    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.909832    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.910091    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.910310    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.910819    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.911547    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sxsxh\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.911766    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.912204    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f4w6c\""
Sep 14 07:49:05.913243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.912477    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 07:49:05.928702 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.928673    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lfwk7"
Sep 14 07:49:05.938241 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.937874    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lfwk7"
Sep 14 07:49:05.957417 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.957231    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:05.987348 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.987281    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.997878    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nntj4\" (UniqueName: \"kubernetes.io/projected/38caa194-e116-44a2-a0d9-294f8841b35d-kube-api-access-nntj4\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.997921    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/378da416-3160-4554-99c3-9f79d9b34aab-tmp-dir\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.997951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nrsj\" (UniqueName: \"kubernetes.io/projected/618ca54d-af1f-4f80-a9e9-e93710b99d5b-kube-api-access-6nrsj\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.997999    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysctl-d\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998116    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysctl-conf\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwbpd\" (UniqueName: \"kubernetes.io/projected/4b39851f-2475-4f3e-b814-a2dbd27f3787-kube-api-access-rwbpd\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-sys-fs\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998226    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq57h\" (UniqueName: \"kubernetes.io/projected/0c653e29-6c65-45ff-a54a-365e694f5e5f-kube-api-access-cq57h\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998273    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-ovnkube-script-lib\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998297    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/378da416-3160-4554-99c3-9f79d9b34aab-tmp-dir\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998311    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysctl-d\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/378da416-3160-4554-99c3-9f79d9b34aab-hosts-file\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998362    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/378da416-3160-4554-99c3-9f79d9b34aab-hosts-file\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998392    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:05.998468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998431    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-system-cni-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.998515    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-socket-dir-parent\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysctl-conf\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:05.998588    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:06.498565704 +0000 UTC m=+2.997958460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-multus-certs\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998632    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c653e29-6c65-45ff-a54a-365e694f5e5f-host-slash\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998677    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998703    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-system-cni-dir\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998735    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-host\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-cni-multus\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998807    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-systemd\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998830    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-host\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998833    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp2mp\" (UniqueName: \"kubernetes.io/projected/5fb50d7f-b3c1-4169-a405-ece674e43bfc-kube-api-access-zp2mp\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998864    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05bfe619-feda-4437-a91f-4a26bcf42615-agent-certs\") pod \"konnectivity-agent-wrck8\" (UID: \"05bfe619-feda-4437-a91f-4a26bcf42615\") " pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-kubernetes\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:05.999333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.998933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-etc-selinux\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-kubernetes\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999134    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999198    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-modprobe-d\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-cni-bin\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999278    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-daemon-config\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999302    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-systemd-units\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999336    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999361    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-run-netns\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999388    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-modprobe-d\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999393    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-etc-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999450    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60928bfa-f870-48b6-9806-1975c0990803-kubelet-config\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999483    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-lib-modules\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999509    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-hostroot\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999533    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-kubelet\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.000293 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999558    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/618ca54d-af1f-4f80-a9e9-e93710b99d5b-host\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999588    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05bfe619-feda-4437-a91f-4a26bcf42615-konnectivity-ca\") pod \"konnectivity-agent-wrck8\" (UID: \"05bfe619-feda-4437-a91f-4a26bcf42615\") " pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999614    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-var-lib-kubelet\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999638    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-k8s-cni-cncf-io\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999649    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-lib-modules\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999678    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/618ca54d-af1f-4f80-a9e9-e93710b99d5b-host\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999707    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c653e29-6c65-45ff-a54a-365e694f5e5f-iptables-alerter-script\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999759    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-var-lib-kubelet\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-tuned\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999837    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-cnibin\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999866    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-conf-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999891    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-etc-kubernetes\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999915    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-node-log\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999948    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cnibin\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:05.999975    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000036    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nq2m\" (UniqueName: \"kubernetes.io/projected/be3af394-a1c5-4410-be29-eff8e1a01aa5-kube-api-access-6nq2m\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000083    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-device-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.001166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000107    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-cni-netd\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000131    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-env-overrides\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cni-binary-copy\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-sys\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b39851f-2475-4f3e-b814-a2dbd27f3787-tmp\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-kubelet\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000264    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sf24\" (UniqueName: \"kubernetes.io/projected/72b04ab9-53df-4a0d-bf76-bc4556235b0c-kube-api-access-2sf24\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05bfe619-feda-4437-a91f-4a26bcf42615-konnectivity-ca\") pod \"konnectivity-agent-wrck8\" (UID: \"05bfe619-feda-4437-a91f-4a26bcf42615\") " pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000294    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-var-lib-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000320    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-ovnkube-config\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000365    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-run\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-cni-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000438    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72b04ab9-53df-4a0d-bf76-bc4556235b0c-cni-binary-copy\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000643    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000666    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-sys\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-systemd\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000730    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:06.002058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000736    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-run\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000751    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60928bfa-f870-48b6-9806-1975c0990803-dbus\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysconfig\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-registration-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000835    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqsvn\" (UniqueName: \"kubernetes.io/projected/09945329-009a-4118-80a5-0c135f12756b-kube-api-access-fqsvn\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000880    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-log-socket\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000916    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38caa194-e116-44a2-a0d9-294f8841b35d-ovn-node-metrics-cert\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-netns\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-ovn\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.000990    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-cni-bin\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001012    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001053    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfqx5\" (UniqueName: \"kubernetes.io/projected/378da416-3160-4554-99c3-9f79d9b34aab-kube-api-access-wfqx5\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001076    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/618ca54d-af1f-4f80-a9e9-e93710b99d5b-serviceca\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-socket-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001120    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-slash\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.002919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001256    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-systemd\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.004075 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001312    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-sysconfig\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.004075 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.001911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/618ca54d-af1f-4f80-a9e9-e93710b99d5b-serviceca\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:06.004338 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.004315    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05bfe619-feda-4437-a91f-4a26bcf42615-agent-certs\") pod \"konnectivity-agent-wrck8\" (UID: \"05bfe619-feda-4437-a91f-4a26bcf42615\") " pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:06.004695 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.004670    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b39851f-2475-4f3e-b814-a2dbd27f3787-etc-tuned\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.008038 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.007923    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b39851f-2475-4f3e-b814-a2dbd27f3787-tmp\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.012117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.012038    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nrsj\" (UniqueName: \"kubernetes.io/projected/618ca54d-af1f-4f80-a9e9-e93710b99d5b-kube-api-access-6nrsj\") pod \"node-ca-wpgcg\" (UID: \"618ca54d-af1f-4f80-a9e9-e93710b99d5b\") " pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:06.016493 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.016166    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfqx5\" (UniqueName: \"kubernetes.io/projected/378da416-3160-4554-99c3-9f79d9b34aab-kube-api-access-wfqx5\") pod \"node-resolver-ptgdj\" (UID: \"378da416-3160-4554-99c3-9f79d9b34aab\") " pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:06.020288 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.020196    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal" event={"ID":"6b56fcbc9aa34740ef36ad85c9970517","Type":"ContainerStarted","Data":"d8fd8c8a59e958dbebeab01979182c737dd1dc9dae408be24c90537410ccd3b0"}
Sep 14 07:49:06.020893 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.020797    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwbpd\" (UniqueName: \"kubernetes.io/projected/4b39851f-2475-4f3e-b814-a2dbd27f3787-kube-api-access-rwbpd\") pod \"tuned-slx9r\" (UID: \"4b39851f-2475-4f3e-b814-a2dbd27f3787\") " pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.022309 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.022279    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" event={"ID":"e9041db37f31c68c35286bb7d3caee1d","Type":"ContainerStarted","Data":"7d42defcc0d67d831b4689251f9f1a0debae3368101d2b1bf66472c74b28d35a"}
Sep 14 07:49:06.023556 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.023510    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nq2m\" (UniqueName: \"kubernetes.io/projected/be3af394-a1c5-4410-be29-eff8e1a01aa5-kube-api-access-6nq2m\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:06.102126 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102085    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqsvn\" (UniqueName: \"kubernetes.io/projected/09945329-009a-4118-80a5-0c135f12756b-kube-api-access-fqsvn\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.102126 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102132    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-log-socket\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102369 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102201    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-log-socket\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102369 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38caa194-e116-44a2-a0d9-294f8841b35d-ovn-node-metrics-cert\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102369 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102305    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-netns\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.102369 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102331    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-ovn\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102369 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102355    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-cni-bin\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102416    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-socket-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102441    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102453    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-ovn\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-slash\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nntj4\" (UniqueName: \"kubernetes.io/projected/38caa194-e116-44a2-a0d9-294f8841b35d-kube-api-access-nntj4\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102512    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-slash\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102559    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-cni-bin\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.102641 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102598    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-netns\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102805    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-sys-fs\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102837    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq57h\" (UniqueName: \"kubernetes.io/projected/0c653e29-6c65-45ff-a54a-365e694f5e5f-kube-api-access-cq57h\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102861    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-ovnkube-script-lib\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-system-cni-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102915    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-socket-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.102928    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-socket-dir-parent\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-sys-fs\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103127    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103144    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-multus-certs\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103159 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103156    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-system-cni-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103184    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c653e29-6c65-45ff-a54a-365e694f5e5f-host-slash\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103198    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-socket-dir-parent\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103213    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103218    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103242    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-system-cni-dir\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103260    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c653e29-6c65-45ff-a54a-365e694f5e5f-host-slash\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103277    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-multus-certs\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-cni-multus\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103368    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-system-cni-dir\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-systemd\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103406    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-cni-multus\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp2mp\" (UniqueName: \"kubernetes.io/projected/5fb50d7f-b3c1-4169-a405-ece674e43bfc-kube-api-access-zp2mp\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-etc-selinux\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103583    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.103674 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103614    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-cni-bin\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103641    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-daemon-config\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.103647    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103701    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-systemd-units\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103710    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-cni-bin\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103729    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-run-netns\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103753    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-systemd\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103757    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-etc-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103766    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-run-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103782    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60928bfa-f870-48b6-9806-1975c0990803-kubelet-config\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.103831    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:06.603811001 +0000 UTC m=+3.103203781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103877    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-ovnkube-script-lib\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103891    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-etc-selinux\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103929    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103968    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-run-netns\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.103970    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-etc-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104044    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60928bfa-f870-48b6-9806-1975c0990803-kubelet-config\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.104391 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-hostroot\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104124    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-systemd-units\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104143    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104194    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-kubelet\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104265    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-hostroot\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104275    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-kubelet\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-k8s-cni-cncf-io\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104396    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-run-k8s-cni-cncf-io\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c653e29-6c65-45ff-a54a-365e694f5e5f-iptables-alerter-script\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104442    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-daemon-config\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104456    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-cnibin\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-conf-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104528    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-cnibin\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104534    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-etc-kubernetes\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104564    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-etc-kubernetes\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-node-log\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104624    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-conf-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cnibin\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.105273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104683    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104714    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cnibin\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104751    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-node-log\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104787    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-device-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104820    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-cni-netd\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104846    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-env-overrides\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104888    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cni-binary-copy\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104901    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-cni-netd\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-kubelet\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104954    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sf24\" (UniqueName: \"kubernetes.io/projected/72b04ab9-53df-4a0d-bf76-bc4556235b0c-kube-api-access-2sf24\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.104953    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-device-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105121    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c653e29-6c65-45ff-a54a-365e694f5e5f-iptables-alerter-script\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-var-lib-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-ovnkube-config\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105251    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-cni-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72b04ab9-53df-4a0d-bf76-bc4556235b0c-cni-binary-copy\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-var-lib-openvswitch\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.105862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105308    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105368    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60928bfa-f870-48b6-9806-1975c0990803-dbus\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105382    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-multus-cni-dir\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105401    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-registration-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-cni-binary-copy\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105489    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09945329-009a-4118-80a5-0c135f12756b-registration-dir\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105251    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72b04ab9-53df-4a0d-bf76-bc4556235b0c-host-var-lib-kubelet\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38caa194-e116-44a2-a0d9-294f8841b35d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-env-overrides\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105652    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60928bfa-f870-48b6-9806-1975c0990803-dbus\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105212    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5fb50d7f-b3c1-4169-a405-ece674e43bfc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.105935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38caa194-e116-44a2-a0d9-294f8841b35d-ovnkube-config\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.106056    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72b04ab9-53df-4a0d-bf76-bc4556235b0c-cni-binary-copy\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.106685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.106487    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38caa194-e116-44a2-a0d9-294f8841b35d-ovn-node-metrics-cert\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.110355 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.110307    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nntj4\" (UniqueName: \"kubernetes.io/projected/38caa194-e116-44a2-a0d9-294f8841b35d-kube-api-access-nntj4\") pod \"ovnkube-node-cth69\" (UID: \"38caa194-e116-44a2-a0d9-294f8841b35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.110624 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.110597    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:06.110624 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.110620    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:06.110829 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.110631    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:06.110829 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.110686    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:06.610670617 +0000 UTC m=+3.110063378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:06.111224 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.111206    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqsvn\" (UniqueName: \"kubernetes.io/projected/09945329-009a-4118-80a5-0c135f12756b-kube-api-access-fqsvn\") pod \"aws-ebs-csi-driver-node-lvsdv\" (UID: \"09945329-009a-4118-80a5-0c135f12756b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.111720 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.111694    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq57h\" (UniqueName: \"kubernetes.io/projected/0c653e29-6c65-45ff-a54a-365e694f5e5f-kube-api-access-cq57h\") pod \"iptables-alerter-glvzr\" (UID: \"0c653e29-6c65-45ff-a54a-365e694f5e5f\") " pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.112421 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.112404    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp2mp\" (UniqueName: \"kubernetes.io/projected/5fb50d7f-b3c1-4169-a405-ece674e43bfc-kube-api-access-zp2mp\") pod \"multus-additional-cni-plugins-hm6hk\" (UID: \"5fb50d7f-b3c1-4169-a405-ece674e43bfc\") " pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.113129 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.113112    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sf24\" (UniqueName: \"kubernetes.io/projected/72b04ab9-53df-4a0d-bf76-bc4556235b0c-kube-api-access-2sf24\") pod \"multus-jn2x4\" (UID: \"72b04ab9-53df-4a0d-bf76-bc4556235b0c\") " pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.194082 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.193996    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:06.206527 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.206501    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-slx9r"
Sep 14 07:49:06.218609 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.218584    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-glvzr"
Sep 14 07:49:06.228460 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.228433    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ptgdj"
Sep 14 07:49:06.237493 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.237474    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wpgcg"
Sep 14 07:49:06.251371 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.251343    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv"
Sep 14 07:49:06.259347 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.259328    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:06.267196 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.267177    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hm6hk"
Sep 14 07:49:06.272070 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.272049    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jn2x4"
Sep 14 07:49:06.507571 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.507466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:06.507748 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.507612    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:06.507748 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.507695    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:07.507678635 +0000 UTC m=+4.007071392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:06.608399 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.608363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:06.608602 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.608506    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:06.608602 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.608570    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:07.608551243 +0000 UTC m=+4.107944002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:06.709448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.709409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:06.709602 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.709584    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:06.709656 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.709607    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:06.709656 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.709617    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:06.709729 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:06.709682    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:07.709658591 +0000 UTC m=+4.209051366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:06.941283 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.941192    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:44:06 +0000 UTC" deadline="2028-03-31 03:48:52.266018916 +0000 UTC"
Sep 14 07:49:06.941283 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:06.941227    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13531h59m45.324796324s"
Sep 14 07:49:07.013046 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.013006    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:07.013265 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.013010    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:07.013265 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.013151    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:07.013380 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.013259    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:07.305430 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:07.305156    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b04ab9_53df_4a0d_bf76_bc4556235b0c.slice/crio-ce6126865692747860bc8a74e90a6204399d80ac52ade0e6b7f6b9dc82fb8d1f WatchSource:0}: Error finding container ce6126865692747860bc8a74e90a6204399d80ac52ade0e6b7f6b9dc82fb8d1f: Status 404 returned error can't find the container with id ce6126865692747860bc8a74e90a6204399d80ac52ade0e6b7f6b9dc82fb8d1f
Sep 14 07:49:07.307549 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:07.307441    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b39851f_2475_4f3e_b814_a2dbd27f3787.slice/crio-3593300880c5c889f4f5e38f1bee5f9c2f0ab79a93f12e77febbf70539145e05 WatchSource:0}: Error finding container 3593300880c5c889f4f5e38f1bee5f9c2f0ab79a93f12e77febbf70539145e05: Status 404 returned error can't find the container with id 3593300880c5c889f4f5e38f1bee5f9c2f0ab79a93f12e77febbf70539145e05
Sep 14 07:49:07.311708 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:07.311605    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c653e29_6c65_45ff_a54a_365e694f5e5f.slice/crio-eea1039db9b66f4aaa33a412adb28701963441b1ce260d401ac99aa60777ad78 WatchSource:0}: Error finding container eea1039db9b66f4aaa33a412adb28701963441b1ce260d401ac99aa60777ad78: Status 404 returned error can't find the container with id eea1039db9b66f4aaa33a412adb28701963441b1ce260d401ac99aa60777ad78
Sep 14 07:49:07.313207 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:07.313120    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38caa194_e116_44a2_a0d9_294f8841b35d.slice/crio-42804183c3e7daa8a3917fc26d4659dbdccbd1ad63ba5c718d7e146a45200147 WatchSource:0}: Error finding container 42804183c3e7daa8a3917fc26d4659dbdccbd1ad63ba5c718d7e146a45200147: Status 404 returned error can't find the container with id 42804183c3e7daa8a3917fc26d4659dbdccbd1ad63ba5c718d7e146a45200147
Sep 14 07:49:07.516770 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.516733    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:07.516953 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.516920    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:07.517056 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.517017    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:09.516994497 +0000 UTC m=+6.016387273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:07.617550 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.617519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:07.617703 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.617688    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:07.617768 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.617758    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:09.617732544 +0000 UTC m=+6.117125301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:07.719260 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.718979    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:07.719260 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.719222    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:07.719260 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.719255    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:07.719260 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.719268    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:07.719613 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:07.719338    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:09.719320041 +0000 UTC m=+6.218712817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:07.942363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.942226    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:44:06 +0000 UTC" deadline="2028-05-05 17:30:11.100232142 +0000 UTC"
Sep 14 07:49:07.942363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:07.942288    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14385h41m3.157948228s"
Sep 14 07:49:08.015986 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.015195    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:08.015986 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:08.015323    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:08.044062 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.043364    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal" event={"ID":"6b56fcbc9aa34740ef36ad85c9970517","Type":"ContainerStarted","Data":"b1205d79a8842b9879cf2b5454ee17ab6a8f835e26564b942a6de06486a7e882"}
Sep 14 07:49:08.060521 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.060454    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal" podStartSLOduration=2.060433603 podStartE2EDuration="2.060433603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:08.060245111 +0000 UTC m=+4.559637896" watchObservedRunningTime="2026-09-14 07:49:08.060433603 +0000 UTC m=+4.559826386"
Sep 14 07:49:08.062623 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.062590    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerStarted","Data":"268478bfc7c87c88a23657bc756a386a529f82e3cb03959d2f0ec3c298e003a9"}
Sep 14 07:49:08.069339 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.069301    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-glvzr" event={"ID":"0c653e29-6c65-45ff-a54a-365e694f5e5f","Type":"ContainerStarted","Data":"eea1039db9b66f4aaa33a412adb28701963441b1ce260d401ac99aa60777ad78"}
Sep 14 07:49:08.079092 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.079001    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jn2x4" event={"ID":"72b04ab9-53df-4a0d-bf76-bc4556235b0c","Type":"ContainerStarted","Data":"ce6126865692747860bc8a74e90a6204399d80ac52ade0e6b7f6b9dc82fb8d1f"}
Sep 14 07:49:08.088096 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.087926    2556 generic.go:356] "Generic (PLEG): container finished" podID="e9041db37f31c68c35286bb7d3caee1d" containerID="486442c929c3466f482f2037355815de638f6d02a8500f73ac60b6aeaacad553" exitCode=0
Sep 14 07:49:08.088096 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.088004    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" event={"ID":"e9041db37f31c68c35286bb7d3caee1d","Type":"ContainerDied","Data":"486442c929c3466f482f2037355815de638f6d02a8500f73ac60b6aeaacad553"}
Sep 14 07:49:08.102702 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.102672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ptgdj" event={"ID":"378da416-3160-4554-99c3-9f79d9b34aab","Type":"ContainerStarted","Data":"16d81c87927dcf32487c34f67f7fd5b83bc6efe8f67cd6be7297263d775dc47d"}
Sep 14 07:49:08.112613 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.112583    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wrck8" event={"ID":"05bfe619-feda-4437-a91f-4a26bcf42615","Type":"ContainerStarted","Data":"26556c8b3283586e7bdb83c55d0ced5c84d29d9c91972af7a2a70ae5c494cf6f"}
Sep 14 07:49:08.119232 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.119204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wpgcg" event={"ID":"618ca54d-af1f-4f80-a9e9-e93710b99d5b","Type":"ContainerStarted","Data":"3bddba53d2fb829711b3d0a119ad4e832b54b7f00f88f28bff4719bd576a4dc8"}
Sep 14 07:49:08.125158 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.125071    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv" event={"ID":"09945329-009a-4118-80a5-0c135f12756b","Type":"ContainerStarted","Data":"64476f4a8d1c583acda7b31aade31352f4649990017bf69bb6ba0141c701bf1c"}
Sep 14 07:49:08.127594 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.127568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"42804183c3e7daa8a3917fc26d4659dbdccbd1ad63ba5c718d7e146a45200147"}
Sep 14 07:49:08.139171 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:08.139143    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-slx9r" event={"ID":"4b39851f-2475-4f3e-b814-a2dbd27f3787","Type":"ContainerStarted","Data":"3593300880c5c889f4f5e38f1bee5f9c2f0ab79a93f12e77febbf70539145e05"}
Sep 14 07:49:09.013247 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:09.012415    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:09.013247 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.012560    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:09.013247 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:09.013019    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:09.013247 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.013136    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:09.166153 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:09.166110    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" event={"ID":"e9041db37f31c68c35286bb7d3caee1d","Type":"ContainerStarted","Data":"c6f8fa017ddf87f7194bd6fa9a4924f034ba255ae9944ae30a8efe2e4fff9182"}
Sep 14 07:49:09.534192 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:09.533536    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:09.534192 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.533750    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:09.534192 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.533816    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:13.533796635 +0000 UTC m=+10.033189401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:09.634535 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:09.634493    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:09.634879 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.634855    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:09.634980 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.634939    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:13.63491637 +0000 UTC m=+10.134309133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:09.735655 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:09.735614    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:09.735836 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.735794    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:09.735836 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.735814    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:09.735836 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.735827    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:09.736004 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:09.735890    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:13.735871193 +0000 UTC m=+10.235263953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:10.013820 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:10.013194    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:10.013820 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:10.013340    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:11.013792 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:11.013310    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:11.013792 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:11.013438    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:11.013792 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:11.013609    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:11.013792 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:11.013742    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:12.013150 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:12.013107    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:12.013332 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:12.013267    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:13.013209 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:13.012544    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:13.013209 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:13.012539    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:13.013209 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.012692    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:13.013209 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.012813    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:13.570565 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:13.570524    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:13.570754 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.570698    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:13.570816 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.570772    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:21.57074935 +0000 UTC m=+18.070142123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:13.671299 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:13.671262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:13.671504 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.671424    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:13.671504 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.671497    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:21.671481092 +0000 UTC m=+18.170873850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:13.773031 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:13.772397    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:13.773031 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.772599    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:13.773031 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.772618    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:13.773031 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.772630    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:13.773031 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:13.772683    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:21.772667504 +0000 UTC m=+18.272060274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:14.014057 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:14.013955    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:14.014478 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:14.014089    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:15.012523 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:15.012483    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:15.012722 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:15.012536    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:15.012722 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:15.012626    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:15.012819 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:15.012749    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:16.013318 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:16.013238    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:16.013835 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:16.013360    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:17.012425 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:17.012388    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:17.012589 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:17.012388    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:17.012589 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:17.012514    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:17.012675 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:17.012650    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:18.012568 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:18.012529    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:18.012946 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:18.012644    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:19.012884 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:19.012850    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:19.013288 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:19.012995    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:19.013288 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:19.013054    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:19.013288 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:19.013172    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:20.012603 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:20.012359    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:20.012763 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:20.012716    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:21.012568 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:21.012536    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:21.013098 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.012664    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:21.013098 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:21.012712    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:21.013098 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.012817    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:21.630317 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:21.630285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:21.630592 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.630448    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:21.630592 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.630524    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:37.630504582 +0000 UTC m=+34.129897345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:21.731514 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:21.731473    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:21.731666 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.731614    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:21.731711 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.731672    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:37.731657874 +0000 UTC m=+34.231050629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:21.832372 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:21.832339    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:21.832529 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.832489    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:21.832529 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.832508    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:21.832529 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.832518    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:21.832642 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:21.832566    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:37.832552519 +0000 UTC m=+34.331945275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:22.013362 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:22.013336    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:22.013690 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:22.013431    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:23.012512 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:23.012465    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:23.012708 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:23.012605    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:23.012708 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:23.012668    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:23.012810 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:23.012761    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:24.013260 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:24.013225    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:24.013701 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:24.013336    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:25.012992 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:25.012943    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:25.013177 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:25.013077    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:25.013177 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:25.013136    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:25.013276 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:25.013240    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:26.012648 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.012423    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:26.012784 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:26.012762    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:26.209425 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.209342    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ptgdj" event={"ID":"378da416-3160-4554-99c3-9f79d9b34aab","Type":"ContainerStarted","Data":"8100207260bcfab2d67053a428f081c7962d3b963e9fbb585ddb2265c4640e20"}
Sep 14 07:49:26.210803 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.210772    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wrck8" event={"ID":"05bfe619-feda-4437-a91f-4a26bcf42615","Type":"ContainerStarted","Data":"c65a1eefcca3dd576d93cfca0e21fc49fa560e87ee45ce0ac028aadde2e1db67"}
Sep 14 07:49:26.212179 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.212159    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wpgcg" event={"ID":"618ca54d-af1f-4f80-a9e9-e93710b99d5b","Type":"ContainerStarted","Data":"0a9cdd507eb4eb3776641dbaebf0afc909e53d8fb257019d248fbaffc5890224"}
Sep 14 07:49:26.213573 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.213547    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv" event={"ID":"09945329-009a-4118-80a5-0c135f12756b","Type":"ContainerStarted","Data":"68406db35974520985b39513e64847e836dd91d60facf481a1c61fad9ec52859"}
Sep 14 07:49:26.216464 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.216443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"82c4c39e56447396162e667366378a0ada359fdfef7c94291fa900dd8552cfe1"}
Sep 14 07:49:26.216554 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.216507    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"0bd8251969bbb28f79feb76d6a92e578dd03341ed2c3fc27cd140adb39bb6b23"}
Sep 14 07:49:26.216606 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.216557    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"968c279e292a8b7d080a0db9f5baa1dc26a36d415182bf10aeffc08124ce84e9"}
Sep 14 07:49:26.216642 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.216617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"9dfc27d713e662afeb08e3e0c40f8cbd757418f05b18c7b1af99c246458b160b"}
Sep 14 07:49:26.216682 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.216668    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"f32ad74dbe3470a7d8b4b97c55bf7d212f27a331ca403927e8472df18c4c232d"}
Sep 14 07:49:26.216731 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.216705    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"6c5992eb31781bad8f1d590c7dce05a9e56915d7ec6943ed1508ca054065857a"}
Sep 14 07:49:26.217934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.217907    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-slx9r" event={"ID":"4b39851f-2475-4f3e-b814-a2dbd27f3787","Type":"ContainerStarted","Data":"55bcf0725a17276bee9a0c473b5140f6af0f2c0a133217ccf1cc6c3675fad524"}
Sep 14 07:49:26.219402 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.219381    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb50d7f-b3c1-4169-a405-ece674e43bfc" containerID="f3b8971a33a6d5561b098ea83aec992a0fc3db647273039250467c1d6b0db39b" exitCode=0
Sep 14 07:49:26.219479 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.219443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerDied","Data":"f3b8971a33a6d5561b098ea83aec992a0fc3db647273039250467c1d6b0db39b"}
Sep 14 07:49:26.220835 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.220819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jn2x4" event={"ID":"72b04ab9-53df-4a0d-bf76-bc4556235b0c","Type":"ContainerStarted","Data":"8523905309d68eb8b9ad54506d185d75a70d83ae0ea4a5255ce13095222d0df0"}
Sep 14 07:49:26.224238 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.224192    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ptgdj" podStartSLOduration=4.119461772 podStartE2EDuration="22.224177692s" totalImagesPullingTime="18.10471592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.224121644 +0000 UTC m=+22.723514432" watchObservedRunningTime="2026-09-14 07:49:26.224177692 +0000 UTC m=+22.723570470"
Sep 14 07:49:26.224815 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.224788    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" podStartSLOduration=20.224777933 podStartE2EDuration="20.224777933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:09.182627261 +0000 UTC m=+5.682020062" watchObservedRunningTime="2026-09-14 07:49:26.224777933 +0000 UTC m=+22.724170711"
Sep 14 07:49:26.240532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.240499    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wpgcg" podStartSLOduration=4.131106169 podStartE2EDuration="22.240486575s" totalImagesPullingTime="18.109380406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.240212916 +0000 UTC m=+22.739605693" watchObservedRunningTime="2026-09-14 07:49:26.240486575 +0000 UTC m=+22.739879353"
Sep 14 07:49:26.268375 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.268326    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wrck8" podStartSLOduration=4.163027826 podStartE2EDuration="22.268313456s" totalImagesPullingTime="18.10528563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.253455225 +0000 UTC m=+22.752848004" watchObservedRunningTime="2026-09-14 07:49:26.268313456 +0000 UTC m=+22.767706235"
Sep 14 07:49:26.268533 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:26.268509    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-slx9r" podStartSLOduration=4.126136234 podStartE2EDuration="22.268500563s" totalImagesPullingTime="18.142364329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.267995838 +0000 UTC m=+22.767388633" watchObservedRunningTime="2026-09-14 07:49:26.268500563 +0000 UTC m=+22.767893345"
Sep 14 07:49:27.013098 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.013032    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:27.013098 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.013070    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:27.013334 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:27.013136    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:27.013334 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:27.013245    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:27.183773 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.183588    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 07:49:27.224579 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.224539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-glvzr" event={"ID":"0c653e29-6c65-45ff-a54a-365e694f5e5f","Type":"ContainerStarted","Data":"ba829864a85a35e31726dbc7a886e38fb80a9d58210846e05bfa1670d54a47a1"}
Sep 14 07:49:27.227056 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.226991    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv" event={"ID":"09945329-009a-4118-80a5-0c135f12756b","Type":"ContainerStarted","Data":"b6e6667935da5cf2412993be973c04dc0ed1f5e7924d98549c72bc2df5a0a8f5"}
Sep 14 07:49:27.239117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.239071    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-glvzr" podStartSLOduration=4.121192595 podStartE2EDuration="22.239056873s" totalImagesPullingTime="18.117864278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:27.238869744 +0000 UTC m=+23.738262542" watchObservedRunningTime="2026-09-14 07:49:27.239056873 +0000 UTC m=+23.738449650"
Sep 14 07:49:27.239527 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.239501    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jn2x4" podStartSLOduration=4.065025015 podStartE2EDuration="22.239494498s" totalImagesPullingTime="18.174469483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.31389752 +0000 UTC m=+22.813290298" watchObservedRunningTime="2026-09-14 07:49:27.239494498 +0000 UTC m=+23.738887588"
Sep 14 07:49:27.958608 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.958480    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T07:49:27.183767873Z","UUID":"61f2d3bc-69e5-40f0-8db0-c1ee761baff6","Handler":null,"Name":"","Endpoint":""}
Sep 14 07:49:27.960221 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.960192    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 07:49:27.960339 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:27.960229    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 07:49:28.013706 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:28.013435    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:28.013706 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:28.013551    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:28.233397 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:28.233361    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"d6dee00b6fa83d3d7476642c13e372f780e8552a7bd9020f9a949fcaa00a1f29"}
Sep 14 07:49:29.013050 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:29.013021    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:29.013263 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:29.013022    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:29.013263 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:29.013144    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:29.013263 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:29.013217    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:29.238793 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:29.238713    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv" event={"ID":"09945329-009a-4118-80a5-0c135f12756b","Type":"ContainerStarted","Data":"bb7c707c6cf0f23c34252d0bc9de88ec746ad773511066bb05a0f943d209ab87"}
Sep 14 07:49:29.253895 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:29.253835    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvsdv" podStartSLOduration=4.746965138 podStartE2EDuration="25.253819773s" totalImagesPullingTime="20.506854635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:29.253687661 +0000 UTC m=+25.753080455" watchObservedRunningTime="2026-09-14 07:49:29.253819773 +0000 UTC m=+25.753212550"
Sep 14 07:49:29.397524 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:29.397449    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:30.012736 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:30.012688    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:30.012904 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:30.012825    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:30.355411 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:30.355329    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:30.356030 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:30.356008    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:31.012957 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.012698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:31.013103 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.012707    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:31.013103 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:31.013055    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:31.013236 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:31.013215    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:31.247395 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.247361    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" event={"ID":"38caa194-e116-44a2-a0d9-294f8841b35d","Type":"ContainerStarted","Data":"f51ab08e59d4d683625fc7f944748169bfd5bd907161ec64ddf69027772a055f"}
Sep 14 07:49:31.247578 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.247564    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:31.249339 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.249311    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb50d7f-b3c1-4169-a405-ece674e43bfc" containerID="05bd412200be3d80c7b8b49609a407561a5b0441e1ee5892e7f8b6d57c6ea80e" exitCode=0
Sep 14 07:49:31.249485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.249401    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerDied","Data":"05bd412200be3d80c7b8b49609a407561a5b0441e1ee5892e7f8b6d57c6ea80e"}
Sep 14 07:49:31.250546 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.250048    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wrck8"
Sep 14 07:49:31.262469 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.262446    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:31.275678 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:31.275593    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" podStartSLOduration=8.078087142 podStartE2EDuration="26.275579993s" totalImagesPullingTime="18.197492851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:31.274195561 +0000 UTC m=+27.773588358" watchObservedRunningTime="2026-09-14 07:49:31.275579993 +0000 UTC m=+27.774972770"
Sep 14 07:49:32.013556 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.013519    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:32.014270 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:32.013658    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:32.254444 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.254410    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb50d7f-b3c1-4169-a405-ece674e43bfc" containerID="0cfaee94596b5ba8518da4961de7c0f790f7af9a67060258517db0cb1d3373e5" exitCode=0
Sep 14 07:49:32.254601 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.254497    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerDied","Data":"0cfaee94596b5ba8518da4961de7c0f790f7af9a67060258517db0cb1d3373e5"}
Sep 14 07:49:32.254867 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.254848    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:49:32.255519 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.255402    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:32.271885 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.271814    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:32.706331 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.706254    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tc6ts"]
Sep 14 07:49:32.706468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.706374    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:32.706514 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:32.706486    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:32.709859 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.709832    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n5ks4"]
Sep 14 07:49:32.709999 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.709936    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:32.710074 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:32.710047    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:32.710564 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.710540    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k7dvq"]
Sep 14 07:49:32.710665 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:32.710651    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:32.710792 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:32.710769    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:33.259738 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:33.259512    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb50d7f-b3c1-4169-a405-ece674e43bfc" containerID="b0676d622769b361f0e86007f4a2df2ec594849c96789c3ffaf38d10a5603dc0" exitCode=0
Sep 14 07:49:33.259738 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:33.259592    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerDied","Data":"b0676d622769b361f0e86007f4a2df2ec594849c96789c3ffaf38d10a5603dc0"}
Sep 14 07:49:33.260298 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:33.259893    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:49:34.013728 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:34.013701    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:34.013900 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:34.013786    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:34.262470 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:34.262438    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:49:34.736859 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:34.736774    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:49:35.012521 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:35.012454    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:35.012658 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:35.012454    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:35.012658 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:35.012543    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:35.012658 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:35.012625    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:35.275018 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:35.274919    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-cth69" podUID="38caa194-e116-44a2-a0d9-294f8841b35d" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 14 07:49:36.013510 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:36.013479    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:36.013661 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:36.013632    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:37.013358 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:37.013326    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:37.013806 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:37.013335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:37.013806 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.013450    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5ks4" podUID="a08c2323-f74f-491c-be32-03c3d3c22ba1"
Sep 14 07:49:37.013806 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.013530    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc6ts" podUID="60928bfa-f870-48b6-9806-1975c0990803"
Sep 14 07:49:37.643339 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:37.643300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:37.643592 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.643449    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:37.643592 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.643526    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs podName:be3af394-a1c5-4410-be29-eff8e1a01aa5 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:09.643504543 +0000 UTC m=+66.142897300 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs") pod "network-metrics-daemon-k7dvq" (UID: "be3af394-a1c5-4410-be29-eff8e1a01aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:37.743857 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:37.743816    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:37.744103 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.743981    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:37.744103 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.744052    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret podName:60928bfa-f870-48b6-9806-1975c0990803 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:09.744034014 +0000 UTC m=+66.243426773 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret") pod "global-pull-secret-syncer-tc6ts" (UID: "60928bfa-f870-48b6-9806-1975c0990803") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:37.844783 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:37.844748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:37.845014 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.844868    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:37.845014 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.844882    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:37.845014 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.844891    2556 projected.go:196] Error preparing data for projected volume kube-api-access-f4qkj for pod openshift-network-diagnostics/network-check-target-n5ks4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:37.845014 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:37.844940    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj podName:a08c2323-f74f-491c-be32-03c3d3c22ba1 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:09.84492748 +0000 UTC m=+66.344320237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f4qkj" (UniqueName: "kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj") pod "network-check-target-n5ks4" (UID: "a08c2323-f74f-491c-be32-03c3d3c22ba1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:38.012527 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.012499    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:38.012635 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:38.012611    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k7dvq" podUID="be3af394-a1c5-4410-be29-eff8e1a01aa5"
Sep 14 07:49:38.259216 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.259196    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeReady"
Sep 14 07:49:38.259570 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.259307    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 07:49:38.293346 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.293316    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dcf68555b-mswbv"]
Sep 14 07:49:38.327620 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.327285    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9tqc4"]
Sep 14 07:49:38.327620 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.327404    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.329855 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.329832    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 07:49:38.329985 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.329881    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfxc9\""
Sep 14 07:49:38.329985 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.329896    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 07:49:38.330108 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.330002    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 07:49:38.334551 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.334531    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 07:49:38.343247 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.343227    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f2vbf"]
Sep 14 07:49:38.343339 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.343325    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.345762 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.345734    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 07:49:38.345855 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.345764    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 07:49:38.345855 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.345780    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd8p\""
Sep 14 07:49:38.356483 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.356459    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dcf68555b-mswbv"]
Sep 14 07:49:38.356670 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.356489    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9tqc4"]
Sep 14 07:49:38.356670 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.356509    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f2vbf"]
Sep 14 07:49:38.356670 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.356583    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.358774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.358756    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 07:49:38.358881 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.358783    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8nsp\""
Sep 14 07:49:38.358881 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.358851    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 07:49:38.358881 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.358858    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 07:49:38.449338 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449308    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f14a15-357d-45a7-84f8-5a4d10b369c4-tmp-dir\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.449471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.449471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449380    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75598ef5-752e-4665-a34f-0e232ee45d96-cert\") pod \"ingress-canary-f2vbf\" (UID: \"75598ef5-752e-4665-a34f-0e232ee45d96\") " pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.449471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449417    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-certificates\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.449471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-bound-sa-token\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.449657 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449489    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e8da133-cac2-4e53-88d5-705811ed1c41-ca-trust-extracted\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.449657 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449559    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddwv5\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-kube-api-access-ddwv5\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.449657 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449578    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57kts\" (UniqueName: \"kubernetes.io/projected/75598ef5-752e-4665-a34f-0e232ee45d96-kube-api-access-57kts\") pod \"ingress-canary-f2vbf\" (UID: \"75598ef5-752e-4665-a34f-0e232ee45d96\") " pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.449657 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449598    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-image-registry-private-configuration\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.449657 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449615    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-trusted-ca\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.450055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2f14a15-357d-45a7-84f8-5a4d10b369c4-config-volume\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.450055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449691    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.450055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449725    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-installation-pull-secrets\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.450055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.449741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8pbw\" (UniqueName: \"kubernetes.io/projected/c2f14a15-357d-45a7-84f8-5a4d10b369c4-kube-api-access-t8pbw\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.550654 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550570    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f14a15-357d-45a7-84f8-5a4d10b369c4-tmp-dir\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.550654 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550611    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.550654 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550627    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75598ef5-752e-4665-a34f-0e232ee45d96-cert\") pod \"ingress-canary-f2vbf\" (UID: \"75598ef5-752e-4665-a34f-0e232ee45d96\") " pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.550654 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-certificates\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-bound-sa-token\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e8da133-cac2-4e53-88d5-705811ed1c41-ca-trust-extracted\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:38.550750    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddwv5\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-kube-api-access-ddwv5\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550806    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57kts\" (UniqueName: \"kubernetes.io/projected/75598ef5-752e-4665-a34f-0e232ee45d96-kube-api-access-57kts\") pod \"ingress-canary-f2vbf\" (UID: \"75598ef5-752e-4665-a34f-0e232ee45d96\") " pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550835    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-image-registry-private-configuration\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-trusted-ca\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550887    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2f14a15-357d-45a7-84f8-5a4d10b369c4-config-volume\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550909    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-installation-pull-secrets\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.550996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.550956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8pbw\" (UniqueName: \"kubernetes.io/projected/c2f14a15-357d-45a7-84f8-5a4d10b369c4-kube-api-access-t8pbw\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.551491 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:38.551054    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:38.551491 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:38.551078    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dcf68555b-mswbv: secret "image-registry-tls" not found
Sep 14 07:49:38.551491 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:38.551120    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls podName:c2f14a15-357d-45a7-84f8-5a4d10b369c4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:39.051098644 +0000 UTC m=+35.550491401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls") pod "dns-default-9tqc4" (UID: "c2f14a15-357d-45a7-84f8-5a4d10b369c4") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:38.551491 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:38.551342    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls podName:1e8da133-cac2-4e53-88d5-705811ed1c41 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:39.051325425 +0000 UTC m=+35.550718187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls") pod "image-registry-5dcf68555b-mswbv" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41") : secret "image-registry-tls" not found
Sep 14 07:49:38.551491 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.551281    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f14a15-357d-45a7-84f8-5a4d10b369c4-tmp-dir\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.551751 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.551547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-certificates\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.551751 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.551668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e8da133-cac2-4e53-88d5-705811ed1c41-ca-trust-extracted\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.551842 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.551757    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2f14a15-357d-45a7-84f8-5a4d10b369c4-config-volume\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.552166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.552147    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-trusted-ca\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.554944 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.554922    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75598ef5-752e-4665-a34f-0e232ee45d96-cert\") pod \"ingress-canary-f2vbf\" (UID: \"75598ef5-752e-4665-a34f-0e232ee45d96\") " pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.555046 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.554923    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-installation-pull-secrets\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.555046 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.554924    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-image-registry-private-configuration\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.558647 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.558626    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddwv5\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-kube-api-access-ddwv5\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.558747 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.558645    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8pbw\" (UniqueName: \"kubernetes.io/projected/c2f14a15-357d-45a7-84f8-5a4d10b369c4-kube-api-access-t8pbw\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:38.559306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.559289    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-bound-sa-token\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:38.559875 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.559854    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57kts\" (UniqueName: \"kubernetes.io/projected/75598ef5-752e-4665-a34f-0e232ee45d96-kube-api-access-57kts\") pod \"ingress-canary-f2vbf\" (UID: \"75598ef5-752e-4665-a34f-0e232ee45d96\") " pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.665370 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.665343    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f2vbf"
Sep 14 07:49:38.997427 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:38.997355    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f2vbf"]
Sep 14 07:49:39.001932 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:39.001909    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75598ef5_752e_4665_a34f_0e232ee45d96.slice/crio-e5202b30b8f580cac0186055c9dec42a259049d4e65132a64c8b3094c1a9c299 WatchSource:0}: Error finding container e5202b30b8f580cac0186055c9dec42a259049d4e65132a64c8b3094c1a9c299: Status 404 returned error can't find the container with id e5202b30b8f580cac0186055c9dec42a259049d4e65132a64c8b3094c1a9c299
Sep 14 07:49:39.013133 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.013112    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:49:39.013294 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.013276    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:49:39.015299 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.015280    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:49:39.015387 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.015316    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wkwvc\""
Sep 14 07:49:39.016063 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.016043    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.016118 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.016043    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.054237 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.054209    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:39.054339 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.054248    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:39.054378 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:39.054348    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:39.054378 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:39.054354    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:39.054378 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:39.054362    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dcf68555b-mswbv: secret "image-registry-tls" not found
Sep 14 07:49:39.054472 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:39.054402    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls podName:c2f14a15-357d-45a7-84f8-5a4d10b369c4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.054389445 +0000 UTC m=+36.553782200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls") pod "dns-default-9tqc4" (UID: "c2f14a15-357d-45a7-84f8-5a4d10b369c4") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:39.054472 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:39.054422    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls podName:1e8da133-cac2-4e53-88d5-705811ed1c41 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.054411469 +0000 UTC m=+36.553804227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls") pod "image-registry-5dcf68555b-mswbv" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41") : secret "image-registry-tls" not found
Sep 14 07:49:39.273860 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.273766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f2vbf" event={"ID":"75598ef5-752e-4665-a34f-0e232ee45d96","Type":"ContainerStarted","Data":"e5202b30b8f580cac0186055c9dec42a259049d4e65132a64c8b3094c1a9c299"}
Sep 14 07:49:39.276366 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.276340    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb50d7f-b3c1-4169-a405-ece674e43bfc" containerID="32e6779b2daf6817cc3ed179e5631005de3d7fd539b93d1fdd70b917aa86498a" exitCode=0
Sep 14 07:49:39.276490 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.276374    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerDied","Data":"32e6779b2daf6817cc3ed179e5631005de3d7fd539b93d1fdd70b917aa86498a"}
Sep 14 07:49:39.778845 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.778805    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"]
Sep 14 07:49:39.795207 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.795170    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"]
Sep 14 07:49:39.795361 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.795269    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"
Sep 14 07:49:39.797655 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.797628    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.797788 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.797628    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.797909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.797891    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h4nfn\""
Sep 14 07:49:39.859669 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.859623    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv84n\" (UniqueName: \"kubernetes.io/projected/e68e6b4d-e64a-4723-b265-30880b058fdf-kube-api-access-vv84n\") pod \"volume-data-source-validator-86695b7cc7-nstdp\" (UID: \"e68e6b4d-e64a-4723-b265-30880b058fdf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"
Sep 14 07:49:39.960317 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.960275    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv84n\" (UniqueName: \"kubernetes.io/projected/e68e6b4d-e64a-4723-b265-30880b058fdf-kube-api-access-vv84n\") pod \"volume-data-source-validator-86695b7cc7-nstdp\" (UID: \"e68e6b4d-e64a-4723-b265-30880b058fdf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"
Sep 14 07:49:39.968801 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:39.968772    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv84n\" (UniqueName: \"kubernetes.io/projected/e68e6b4d-e64a-4723-b265-30880b058fdf-kube-api-access-vv84n\") pod \"volume-data-source-validator-86695b7cc7-nstdp\" (UID: \"e68e6b4d-e64a-4723-b265-30880b058fdf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"
Sep 14 07:49:40.012719 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.012690    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:49:40.014751 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.014722    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:49:40.014876 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.014779    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfw6g\""
Sep 14 07:49:40.061726 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.061655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:40.061726 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.061715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:40.061901 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:40.061773    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:40.061901 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:40.061791    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dcf68555b-mswbv: secret "image-registry-tls" not found
Sep 14 07:49:40.061901 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:40.061803    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:40.061901 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:40.061847    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls podName:1e8da133-cac2-4e53-88d5-705811ed1c41 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:42.061827152 +0000 UTC m=+38.561219910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls") pod "image-registry-5dcf68555b-mswbv" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41") : secret "image-registry-tls" not found
Sep 14 07:49:40.061901 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:40.061868    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls podName:c2f14a15-357d-45a7-84f8-5a4d10b369c4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:42.061859515 +0000 UTC m=+38.561252274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls") pod "dns-default-9tqc4" (UID: "c2f14a15-357d-45a7-84f8-5a4d10b369c4") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:40.104889 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.104868    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"
Sep 14 07:49:40.282958 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.282919    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb50d7f-b3c1-4169-a405-ece674e43bfc" containerID="239c9b39139a97054cb53524c306b6ea0f18102eba7331083f6d8de9491d3ccf" exitCode=0
Sep 14 07:49:40.283485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.282984    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerDied","Data":"239c9b39139a97054cb53524c306b6ea0f18102eba7331083f6d8de9491d3ccf"}
Sep 14 07:49:40.399694 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:40.399512    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp"]
Sep 14 07:49:40.403428 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:40.403386    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68e6b4d_e64a_4723_b265_30880b058fdf.slice/crio-742e45577e233890651d2f3859733d321a2604e2dac1b6ff186ccc7de9ae0c79 WatchSource:0}: Error finding container 742e45577e233890651d2f3859733d321a2604e2dac1b6ff186ccc7de9ae0c79: Status 404 returned error can't find the container with id 742e45577e233890651d2f3859733d321a2604e2dac1b6ff186ccc7de9ae0c79
Sep 14 07:49:41.290440 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.290406    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" event={"ID":"5fb50d7f-b3c1-4169-a405-ece674e43bfc","Type":"ContainerStarted","Data":"0cb2a02b5e13cbbc612bbdedabf131987245da9f66a496324fb59e50fb56c702"}
Sep 14 07:49:41.291740 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.291715    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp" event={"ID":"e68e6b4d-e64a-4723-b265-30880b058fdf","Type":"ContainerStarted","Data":"742e45577e233890651d2f3859733d321a2604e2dac1b6ff186ccc7de9ae0c79"}
Sep 14 07:49:41.314523 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.314033    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hm6hk" podStartSLOduration=6.9035997049999995 podStartE2EDuration="36.314012445s" totalImagesPullingTime="29.41041274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:41.31284418 +0000 UTC m=+37.812236985" watchObservedRunningTime="2026-09-14 07:49:41.314012445 +0000 UTC m=+37.813405226"
Sep 14 07:49:41.567884 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.567815    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"]
Sep 14 07:49:41.573946 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.573923    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.576251 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.576203    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 14 07:49:41.576251 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.576235    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:41.576492 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.576313    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 14 07:49:41.576492 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.576234    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pbbb5\""
Sep 14 07:49:41.576492 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.576207    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:41.584256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.584230    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"]
Sep 14 07:49:41.675096 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.675050    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61186fd6-9d8f-43b2-967c-9c1752ced815-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.675270 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.675247    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61186fd6-9d8f-43b2-967c-9c1752ced815-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.675345 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.675285    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqmfv\" (UniqueName: \"kubernetes.io/projected/61186fd6-9d8f-43b2-967c-9c1752ced815-kube-api-access-rqmfv\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.776255 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.776221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61186fd6-9d8f-43b2-967c-9c1752ced815-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.776255 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.776260    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqmfv\" (UniqueName: \"kubernetes.io/projected/61186fd6-9d8f-43b2-967c-9c1752ced815-kube-api-access-rqmfv\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.776448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.776292    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61186fd6-9d8f-43b2-967c-9c1752ced815-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.776808 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.776790    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61186fd6-9d8f-43b2-967c-9c1752ced815-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.779709 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.779678    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61186fd6-9d8f-43b2-967c-9c1752ced815-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.786413 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.786387    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqmfv\" (UniqueName: \"kubernetes.io/projected/61186fd6-9d8f-43b2-967c-9c1752ced815-kube-api-access-rqmfv\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wsm58\" (UID: \"61186fd6-9d8f-43b2-967c-9c1752ced815\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:41.887108 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:41.887036    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"
Sep 14 07:49:42.078752 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.078723    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:42.078903 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.078763    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:42.078903 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:42.078862    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:42.078903 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:42.078869    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:42.078903 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:42.078884    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dcf68555b-mswbv: secret "image-registry-tls" not found
Sep 14 07:49:42.079049 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:42.078914    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls podName:c2f14a15-357d-45a7-84f8-5a4d10b369c4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.078901363 +0000 UTC m=+42.578294120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls") pod "dns-default-9tqc4" (UID: "c2f14a15-357d-45a7-84f8-5a4d10b369c4") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:42.079049 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:42.078927    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls podName:1e8da133-cac2-4e53-88d5-705811ed1c41 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.078921486 +0000 UTC m=+42.578314242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls") pod "image-registry-5dcf68555b-mswbv" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41") : secret "image-registry-tls" not found
Sep 14 07:49:42.295694 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.295476    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f2vbf" event={"ID":"75598ef5-752e-4665-a34f-0e232ee45d96","Type":"ContainerStarted","Data":"6563df58a20ddda4d027e8df916d9379c9ddf940cf1612a227a58c43d180ff32"}
Sep 14 07:49:42.297686 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.297662    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp" event={"ID":"e68e6b4d-e64a-4723-b265-30880b058fdf","Type":"ContainerStarted","Data":"64d363211ed9cd30fed74f568e31cebe2b4fe6fbdfaa49aa7df566790f5ea8a8"}
Sep 14 07:49:42.310336 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.310295    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f2vbf" podStartSLOduration=1.170408952 podStartE2EDuration="4.310279115s" totalImagesPullingTime="3.139870163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:42.309130985 +0000 UTC m=+38.808523782" watchObservedRunningTime="2026-09-14 07:49:42.310279115 +0000 UTC m=+38.809671893"
Sep 14 07:49:42.323311 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.323256    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nstdp" podStartSLOduration=1.5837247589999999 podStartE2EDuration="3.323236561s" totalImagesPullingTime="1.739511802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:42.322460999 +0000 UTC m=+38.821853797" watchObservedRunningTime="2026-09-14 07:49:42.323236561 +0000 UTC m=+38.822629341"
Sep 14 07:49:42.407510 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:42.407436    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58"]
Sep 14 07:49:42.410882 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:42.410856    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61186fd6_9d8f_43b2_967c_9c1752ced815.slice/crio-53c4e75fa5f756ea77af871f3158dab9b1d951f3fad59757370311f443b0100a WatchSource:0}: Error finding container 53c4e75fa5f756ea77af871f3158dab9b1d951f3fad59757370311f443b0100a: Status 404 returned error can't find the container with id 53c4e75fa5f756ea77af871f3158dab9b1d951f3fad59757370311f443b0100a
Sep 14 07:49:43.075236 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.075205    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"]
Sep 14 07:49:43.093018 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.092946    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"]
Sep 14 07:49:43.093136 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.093059    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"
Sep 14 07:49:43.094989 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.094956    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-v8lzm\""
Sep 14 07:49:43.187599 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.187567    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pftrc\" (UniqueName: \"kubernetes.io/projected/dcdc7077-343b-4ce4-b8da-ec76d9dbae24-kube-api-access-pftrc\") pod \"network-check-source-6f8b768455-bbzkq\" (UID: \"dcdc7077-343b-4ce4-b8da-ec76d9dbae24\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"
Sep 14 07:49:43.288324 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.288275    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pftrc\" (UniqueName: \"kubernetes.io/projected/dcdc7077-343b-4ce4-b8da-ec76d9dbae24-kube-api-access-pftrc\") pod \"network-check-source-6f8b768455-bbzkq\" (UID: \"dcdc7077-343b-4ce4-b8da-ec76d9dbae24\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"
Sep 14 07:49:43.297145 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.297114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pftrc\" (UniqueName: \"kubernetes.io/projected/dcdc7077-343b-4ce4-b8da-ec76d9dbae24-kube-api-access-pftrc\") pod \"network-check-source-6f8b768455-bbzkq\" (UID: \"dcdc7077-343b-4ce4-b8da-ec76d9dbae24\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"
Sep 14 07:49:43.301830 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.301792    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58" event={"ID":"61186fd6-9d8f-43b2-967c-9c1752ced815","Type":"ContainerStarted","Data":"53c4e75fa5f756ea77af871f3158dab9b1d951f3fad59757370311f443b0100a"}
Sep 14 07:49:43.401425 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.401346    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"
Sep 14 07:49:43.692190 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:43.691865    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq"]
Sep 14 07:49:43.694564 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:43.694535    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcdc7077_343b_4ce4_b8da_ec76d9dbae24.slice/crio-4b896d57df7bb581cf48131a5da8eb4d8589bbe02ded4dce6f2e3af5d6e16dcc WatchSource:0}: Error finding container 4b896d57df7bb581cf48131a5da8eb4d8589bbe02ded4dce6f2e3af5d6e16dcc: Status 404 returned error can't find the container with id 4b896d57df7bb581cf48131a5da8eb4d8589bbe02ded4dce6f2e3af5d6e16dcc
Sep 14 07:49:44.305800 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.305756    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq" event={"ID":"dcdc7077-343b-4ce4-b8da-ec76d9dbae24","Type":"ContainerStarted","Data":"4b896d57df7bb581cf48131a5da8eb4d8589bbe02ded4dce6f2e3af5d6e16dcc"}
Sep 14 07:49:44.684849 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.684771    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4nldc"]
Sep 14 07:49:44.693501 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.693477    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.695755 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.695734    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 07:49:44.696774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.696591    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 07:49:44.696774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.696602    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 07:49:44.696774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.696648    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bqkjn\""
Sep 14 07:49:44.696774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.696629    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 07:49:44.697940 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.697920    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4nldc"]
Sep 14 07:49:44.798781 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.798756    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qsd5\" (UniqueName: \"kubernetes.io/projected/1c530033-9200-49a2-b490-6fce8c932216-kube-api-access-5qsd5\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.798860 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.798797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.798860 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.798817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1c530033-9200-49a2-b490-6fce8c932216-crio-socket\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.798939 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.798870    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c530033-9200-49a2-b490-6fce8c932216-data-volume\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.798988 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.798938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.899882 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.899847    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qsd5\" (UniqueName: \"kubernetes.io/projected/1c530033-9200-49a2-b490-6fce8c932216-kube-api-access-5qsd5\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.900055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.899900    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.900055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.899927    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1c530033-9200-49a2-b490-6fce8c932216-crio-socket\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.900055 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.899951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c530033-9200-49a2-b490-6fce8c932216-data-volume\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.900223 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:44.900073    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:44.900223 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:44.900142    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.400120314 +0000 UTC m=+41.899513087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:44.900223 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.900152    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1c530033-9200-49a2-b490-6fce8c932216-crio-socket\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.900379 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.900228    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.900379 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:44.900294    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:44.900379 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:44.900361    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.400345314 +0000 UTC m=+41.899738075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:44.900503 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.900428    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c530033-9200-49a2-b490-6fce8c932216-data-volume\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:44.909838 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:44.909813    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qsd5\" (UniqueName: \"kubernetes.io/projected/1c530033-9200-49a2-b490-6fce8c932216-kube-api-access-5qsd5\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:45.310628 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:45.310586    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58" event={"ID":"61186fd6-9d8f-43b2-967c-9c1752ced815","Type":"ContainerStarted","Data":"1d932bb1a8f556ed19139dee0a806455a5795332596baae7b7cd9999d3b0b1c8"}
Sep 14 07:49:45.325086 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:45.325035    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58" podStartSLOduration=1.992614203 podStartE2EDuration="4.325014455s" totalImagesPullingTime="2.332400252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:45.324295364 +0000 UTC m=+41.823688142" watchObservedRunningTime="2026-09-14 07:49:45.325014455 +0000 UTC m=+41.824407234"
Sep 14 07:49:45.405021 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:45.404984    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:45.405164 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:45.405043    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:45.405164 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:45.405109    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.405091478 +0000 UTC m=+42.904484256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:45.405253 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:45.405165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:45.405293 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:45.405261    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:45.405339 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:45.405328    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.405311231 +0000 UTC m=+42.904703994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.112416 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:46.112378    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:46.112576 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:46.112533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:46.112576 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.112542    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:46.112671 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.112606    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls podName:c2f14a15-357d-45a7-84f8-5a4d10b369c4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:54.112590808 +0000 UTC m=+50.611983565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls") pod "dns-default-9tqc4" (UID: "c2f14a15-357d-45a7-84f8-5a4d10b369c4") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:46.112671 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.112665    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:46.112755 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.112678    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dcf68555b-mswbv: secret "image-registry-tls" not found
Sep 14 07:49:46.112755 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.112721    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls podName:1e8da133-cac2-4e53-88d5-705811ed1c41 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:54.112709977 +0000 UTC m=+50.612102737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls") pod "image-registry-5dcf68555b-mswbv" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41") : secret "image-registry-tls" not found
Sep 14 07:49:46.415243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:46.415156    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:46.415660 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.415268    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.415660 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:46.415297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:46.415660 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.415327    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:48.415309708 +0000 UTC m=+44.914702484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.415660 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.415364    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:46.415660 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:46.415427    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:48.415409817 +0000 UTC m=+44.914802591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:46.661791 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:46.661762    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ptgdj_378da416-3160-4554-99c3-9f79d9b34aab/dns-node-resolver/0.log"
Sep 14 07:49:47.861312 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:47.861283    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wpgcg_618ca54d-af1f-4f80-a9e9-e93710b99d5b/node-ca/0.log"
Sep 14 07:49:48.261868 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:48.261786    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f2vbf_75598ef5-752e-4665-a34f-0e232ee45d96/serve-healthcheck-canary/0.log"
Sep 14 07:49:48.319281 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:48.319244    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq" event={"ID":"dcdc7077-343b-4ce4-b8da-ec76d9dbae24","Type":"ContainerStarted","Data":"997a4b2927079c92541fab71c9ae9d1c2ec022dacd520e5c3d1fb7c9914b4917"}
Sep 14 07:49:48.334044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:48.333996    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bbzkq" podStartSLOduration=1.135480818 podStartE2EDuration="5.333980961s" totalImagesPullingTime="4.198500143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:48.333421029 +0000 UTC m=+44.832813812" watchObservedRunningTime="2026-09-14 07:49:48.333980961 +0000 UTC m=+44.833373736"
Sep 14 07:49:48.431214 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:48.431185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:48.431355 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:48.431250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:48.431355 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:48.431339    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:48.431355 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:48.431342    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:48.431451 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:48.431398    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:52.43138515 +0000 UTC m=+48.930777906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:48.431451 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:48.431412    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:52.431406088 +0000 UTC m=+48.930798844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:50.964518 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:50.964487    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tb4ks"]
Sep 14 07:49:50.998215 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:50.998187    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tb4ks"]
Sep 14 07:49:50.998356 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:50.998306    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.000536 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.000513    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 14 07:49:51.000673 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.000522    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 14 07:49:51.001180 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.001162    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 14 07:49:51.001275 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.001211    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 14 07:49:51.001336 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.001210    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b9h9n\""
Sep 14 07:49:51.154122 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.154076    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7dbb388c-214c-4202-817a-5e614f31e62f-signing-key\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.154122 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.154115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4rmz\" (UniqueName: \"kubernetes.io/projected/7dbb388c-214c-4202-817a-5e614f31e62f-kube-api-access-m4rmz\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.154320 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.154238    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7dbb388c-214c-4202-817a-5e614f31e62f-signing-cabundle\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.254718 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.254642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7dbb388c-214c-4202-817a-5e614f31e62f-signing-cabundle\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.254846 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.254762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7dbb388c-214c-4202-817a-5e614f31e62f-signing-key\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.254846 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.254793    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4rmz\" (UniqueName: \"kubernetes.io/projected/7dbb388c-214c-4202-817a-5e614f31e62f-kube-api-access-m4rmz\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.255479 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.255454    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7dbb388c-214c-4202-817a-5e614f31e62f-signing-cabundle\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.257207 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.257183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7dbb388c-214c-4202-817a-5e614f31e62f-signing-key\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.262944 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.262924    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4rmz\" (UniqueName: \"kubernetes.io/projected/7dbb388c-214c-4202-817a-5e614f31e62f-kube-api-access-m4rmz\") pod \"service-ca-568869fdc5-tb4ks\" (UID: \"7dbb388c-214c-4202-817a-5e614f31e62f\") " pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.307157 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.307133    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-tb4ks"
Sep 14 07:49:51.586867 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:51.586792    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tb4ks"]
Sep 14 07:49:51.590418 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:49:51.590386    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbb388c_214c_4202_817a_5e614f31e62f.slice/crio-f3b79888b5bd3ad4f287be7616c9aca29ef73f23f8f08fa8c338aa23ae558282 WatchSource:0}: Error finding container f3b79888b5bd3ad4f287be7616c9aca29ef73f23f8f08fa8c338aa23ae558282: Status 404 returned error can't find the container with id f3b79888b5bd3ad4f287be7616c9aca29ef73f23f8f08fa8c338aa23ae558282
Sep 14 07:49:52.331157 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:52.331110    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-tb4ks" event={"ID":"7dbb388c-214c-4202-817a-5e614f31e62f","Type":"ContainerStarted","Data":"f3b79888b5bd3ad4f287be7616c9aca29ef73f23f8f08fa8c338aa23ae558282"}
Sep 14 07:49:52.466359 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:52.466325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:52.466556 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:52.466409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:49:52.466556 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:52.466490    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:52.466556 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:52.466504    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:52.466696 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:52.466569    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:00.466547935 +0000 UTC m=+56.965940696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:52.466696 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:52.466591    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm podName:1c530033-9200-49a2-b490-6fce8c932216 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:00.466580758 +0000 UTC m=+56.965973521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nldc" (UID: "1c530033-9200-49a2-b490-6fce8c932216") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:54.178771 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:54.178735    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:49:54.179210 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:54.178783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:49:54.179210 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:54.178878    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:54.179210 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:54.178882    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:54.179210 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:54.178902    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dcf68555b-mswbv: secret "image-registry-tls" not found
Sep 14 07:49:54.179210 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:54.178942    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls podName:c2f14a15-357d-45a7-84f8-5a4d10b369c4 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:10.178926647 +0000 UTC m=+66.678319405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls") pod "dns-default-9tqc4" (UID: "c2f14a15-357d-45a7-84f8-5a4d10b369c4") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:54.179210 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:49:54.179009    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls podName:1e8da133-cac2-4e53-88d5-705811ed1c41 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:10.178995383 +0000 UTC m=+66.678388157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls") pod "image-registry-5dcf68555b-mswbv" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41") : secret "image-registry-tls" not found
Sep 14 07:49:54.338138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:54.338104    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-tb4ks" event={"ID":"7dbb388c-214c-4202-817a-5e614f31e62f","Type":"ContainerStarted","Data":"83c3048bbeb62aa60ed3d48c5c2328840e4c161f84fc06ad8c825999b89f41fd"}
Sep 14 07:49:54.353828 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:49:54.353774    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-tb4ks" podStartSLOduration=2.046235674 podStartE2EDuration="4.353755136s" totalImagesPullingTime="2.307519462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:54.353548164 +0000 UTC m=+50.852940958" watchObservedRunningTime="2026-09-14 07:49:54.353755136 +0000 UTC m=+50.853147915"
Sep 14 07:50:00.527632 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:00.527585    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:50:00.528193 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:00.527683    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:50:00.528369 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:00.528345    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1c530033-9200-49a2-b490-6fce8c932216-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:50:00.529875 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:00.529853    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1c530033-9200-49a2-b490-6fce8c932216-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nldc\" (UID: \"1c530033-9200-49a2-b490-6fce8c932216\") " pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:50:00.604844 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:00.604811    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4nldc"
Sep 14 07:50:00.878200 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:00.878130    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4nldc"]
Sep 14 07:50:00.883514 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:00.883462    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c530033_9200_49a2_b490_6fce8c932216.slice/crio-a241d10336cbbbf70cf190deba6031e0312766ee21f2ec8bb3a2b655e2df992e WatchSource:0}: Error finding container a241d10336cbbbf70cf190deba6031e0312766ee21f2ec8bb3a2b655e2df992e: Status 404 returned error can't find the container with id a241d10336cbbbf70cf190deba6031e0312766ee21f2ec8bb3a2b655e2df992e
Sep 14 07:50:01.358326 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:01.358291    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nldc" event={"ID":"1c530033-9200-49a2-b490-6fce8c932216","Type":"ContainerStarted","Data":"e2ec1f4ec6d8e8cd7e4f26330bcc3b57b9454956e1676acee0d9df22b6e132c2"}
Sep 14 07:50:01.358482 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:01.358372    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nldc" event={"ID":"1c530033-9200-49a2-b490-6fce8c932216","Type":"ContainerStarted","Data":"a241d10336cbbbf70cf190deba6031e0312766ee21f2ec8bb3a2b655e2df992e"}
Sep 14 07:50:03.367132 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:03.367081    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nldc" event={"ID":"1c530033-9200-49a2-b490-6fce8c932216","Type":"ContainerStarted","Data":"cf2411cf4c101d3f9942ea985ec1bc49d4ed2e13d0e19a9222e14767d0a2dcc6"}
Sep 14 07:50:04.372936 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:04.372895    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nldc" event={"ID":"1c530033-9200-49a2-b490-6fce8c932216","Type":"ContainerStarted","Data":"5ac31b7069e059cd46062a6906929ecb16b0cf630a467474726f35727bd6bbc5"}
Sep 14 07:50:04.388482 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:04.388434    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4nldc" podStartSLOduration=17.699524438 podStartE2EDuration="20.388414874s" totalImagesPullingTime="2.688890436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:04.387727163 +0000 UTC m=+60.887119954" watchObservedRunningTime="2026-09-14 07:50:04.388414874 +0000 UTC m=+60.887807651"
Sep 14 07:50:05.274073 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:05.274046    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cth69"
Sep 14 07:50:09.695346 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.695300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:50:09.697470 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.697449    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:50:09.708288 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.708260    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3af394-a1c5-4410-be29-eff8e1a01aa5-metrics-certs\") pod \"network-metrics-daemon-k7dvq\" (UID: \"be3af394-a1c5-4410-be29-eff8e1a01aa5\") " pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:50:09.725114 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.725091    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfw6g\""
Sep 14 07:50:09.733843 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.733824    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k7dvq"
Sep 14 07:50:09.796052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.796023    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:50:09.798072 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.798051    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:50:09.808304 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.808286    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60928bfa-f870-48b6-9806-1975c0990803-original-pull-secret\") pod \"global-pull-secret-syncer-tc6ts\" (UID: \"60928bfa-f870-48b6-9806-1975c0990803\") " pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:50:09.897219 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.897182    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:50:09.899626 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.899606    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qkj\" (UniqueName: \"kubernetes.io/projected/a08c2323-f74f-491c-be32-03c3d3c22ba1-kube-api-access-f4qkj\") pod \"network-check-target-n5ks4\" (UID: \"a08c2323-f74f-491c-be32-03c3d3c22ba1\") " pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:50:09.923416 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.923385    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc6ts"
Sep 14 07:50:09.929546 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.929526    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wkwvc\""
Sep 14 07:50:09.938173 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:09.938156    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:50:10.025272 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.025191    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k7dvq"]
Sep 14 07:50:10.028461 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:10.028440    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3af394_a1c5_4410_be29_eff8e1a01aa5.slice/crio-591ea85664db4ee7bfd6d370119af6e596a7601dea1af2d87f7218bed811a82f WatchSource:0}: Error finding container 591ea85664db4ee7bfd6d370119af6e596a7601dea1af2d87f7218bed811a82f: Status 404 returned error can't find the container with id 591ea85664db4ee7bfd6d370119af6e596a7601dea1af2d87f7218bed811a82f
Sep 14 07:50:10.200636 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.200602    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:50:10.200792 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.200664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:50:10.202866 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.202843    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f14a15-357d-45a7-84f8-5a4d10b369c4-metrics-tls\") pod \"dns-default-9tqc4\" (UID: \"c2f14a15-357d-45a7-84f8-5a4d10b369c4\") " pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:50:10.203007 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.202988    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"image-registry-5dcf68555b-mswbv\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") " pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:50:10.224800 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.224694    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tc6ts"]
Sep 14 07:50:10.228457 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:10.228429    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60928bfa_f870_48b6_9806_1975c0990803.slice/crio-6321be3d0bfb7a971448b2df7481e7f492f54fddfc900848bb30525ec284687d WatchSource:0}: Error finding container 6321be3d0bfb7a971448b2df7481e7f492f54fddfc900848bb30525ec284687d: Status 404 returned error can't find the container with id 6321be3d0bfb7a971448b2df7481e7f492f54fddfc900848bb30525ec284687d
Sep 14 07:50:10.255456 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.255434    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n5ks4"]
Sep 14 07:50:10.258352 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:10.258328    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08c2323_f74f_491c_be32_03c3d3c22ba1.slice/crio-3c6ef72c7e4057b1c0636f266e39cef455ff40d79bae721a58ebf42d1dc9e2d4 WatchSource:0}: Error finding container 3c6ef72c7e4057b1c0636f266e39cef455ff40d79bae721a58ebf42d1dc9e2d4: Status 404 returned error can't find the container with id 3c6ef72c7e4057b1c0636f266e39cef455ff40d79bae721a58ebf42d1dc9e2d4
Sep 14 07:50:10.391097 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.391067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n5ks4" event={"ID":"a08c2323-f74f-491c-be32-03c3d3c22ba1","Type":"ContainerStarted","Data":"9d4c8fe3e121833f91f4e3215a37cc5aa61df190914ac4abde30be8be1f1a7b3"}
Sep 14 07:50:10.391227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.391112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n5ks4" event={"ID":"a08c2323-f74f-491c-be32-03c3d3c22ba1","Type":"ContainerStarted","Data":"3c6ef72c7e4057b1c0636f266e39cef455ff40d79bae721a58ebf42d1dc9e2d4"}
Sep 14 07:50:10.391227 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.391159    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:50:10.392344 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.392322    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k7dvq" event={"ID":"be3af394-a1c5-4410-be29-eff8e1a01aa5","Type":"ContainerStarted","Data":"591ea85664db4ee7bfd6d370119af6e596a7601dea1af2d87f7218bed811a82f"}
Sep 14 07:50:10.393415 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.393395    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tc6ts" event={"ID":"60928bfa-f870-48b6-9806-1975c0990803","Type":"ContainerStarted","Data":"6321be3d0bfb7a971448b2df7481e7f492f54fddfc900848bb30525ec284687d"}
Sep 14 07:50:10.405102 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.405064    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n5ks4" podStartSLOduration=65.405052147 podStartE2EDuration="1m5.405052147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:10.404278042 +0000 UTC m=+66.903670830" watchObservedRunningTime="2026-09-14 07:50:10.405052147 +0000 UTC m=+66.904444925"
Sep 14 07:50:10.446685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.446662    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfxc9\""
Sep 14 07:50:10.453864 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.453843    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd8p\""
Sep 14 07:50:10.454905 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.454890    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:50:10.462976 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.462949    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:50:10.750014 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.749795    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dcf68555b-mswbv"]
Sep 14 07:50:10.756663 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:10.756631    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8da133_cac2_4e53_88d5_705811ed1c41.slice/crio-3a391506741f86232cd046a5314e838053de81af5c141b8a7ad7e0ca3cb3fb5a WatchSource:0}: Error finding container 3a391506741f86232cd046a5314e838053de81af5c141b8a7ad7e0ca3cb3fb5a: Status 404 returned error can't find the container with id 3a391506741f86232cd046a5314e838053de81af5c141b8a7ad7e0ca3cb3fb5a
Sep 14 07:50:10.770659 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:10.770624    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9tqc4"]
Sep 14 07:50:10.774111 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:10.774084    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f14a15_357d_45a7_84f8_5a4d10b369c4.slice/crio-a657c6600d1275c1a3a080a7b49764b7396e393e739329736756536216430ead WatchSource:0}: Error finding container a657c6600d1275c1a3a080a7b49764b7396e393e739329736756536216430ead: Status 404 returned error can't find the container with id a657c6600d1275c1a3a080a7b49764b7396e393e739329736756536216430ead
Sep 14 07:50:11.398375 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:11.398339    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tqc4" event={"ID":"c2f14a15-357d-45a7-84f8-5a4d10b369c4","Type":"ContainerStarted","Data":"a657c6600d1275c1a3a080a7b49764b7396e393e739329736756536216430ead"}
Sep 14 07:50:11.400202 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:11.400167    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" event={"ID":"1e8da133-cac2-4e53-88d5-705811ed1c41","Type":"ContainerStarted","Data":"274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619"}
Sep 14 07:50:11.400341 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:11.400221    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" event={"ID":"1e8da133-cac2-4e53-88d5-705811ed1c41","Type":"ContainerStarted","Data":"3a391506741f86232cd046a5314e838053de81af5c141b8a7ad7e0ca3cb3fb5a"}
Sep 14 07:50:11.422425 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:11.421554    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" podStartSLOduration=66.421530624 podStartE2EDuration="1m6.421530624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:11.418701661 +0000 UTC m=+67.918094695" watchObservedRunningTime="2026-09-14 07:50:11.421530624 +0000 UTC m=+67.920923405"
Sep 14 07:50:12.406516 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.406473    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k7dvq" event={"ID":"be3af394-a1c5-4410-be29-eff8e1a01aa5","Type":"ContainerStarted","Data":"c328d8bcfe736f0dbd35bd06448ca9339686c20bf59263c82aedc3a1463354ff"}
Sep 14 07:50:12.406956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.406553    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:50:12.406956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.406578    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k7dvq" event={"ID":"be3af394-a1c5-4410-be29-eff8e1a01aa5","Type":"ContainerStarted","Data":"cc21d9abf449ee667525b208a08f9d708be053b8fea010ba6aefd162e3c9f91b"}
Sep 14 07:50:12.425978 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.425906    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k7dvq" podStartSLOduration=67.154049456 podStartE2EDuration="1m8.42588981s" totalImagesPullingTime="1.271840354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:12.42437456 +0000 UTC m=+68.923767362" watchObservedRunningTime="2026-09-14 07:50:12.42588981 +0000 UTC m=+68.925282589"
Sep 14 07:50:12.456006 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.455955    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"]
Sep 14 07:50:12.458887 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.458859    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.461394 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461097    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 07:50:12.461394 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461303    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:50:12.461394 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461332    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:50:12.461597 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461554    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 07:50:12.461597 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461575    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 07:50:12.461827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461804    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:50:12.461992 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.461807    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 07:50:12.476418 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.476395    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"]
Sep 14 07:50:12.519057 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.519010    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.519238 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.519109    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d6d6fd-0047-4fb7-b965-e3f10d928461-additional-service-ca\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.519238 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.519164    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-hub\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.519238 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.519191    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.519428 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.519269    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdlxc\" (UniqueName: \"kubernetes.io/projected/b5d6d6fd-0047-4fb7-b965-e3f10d928461-kube-api-access-wdlxc\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.519428 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.519297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-ca\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.621004 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.620943    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdlxc\" (UniqueName: \"kubernetes.io/projected/b5d6d6fd-0047-4fb7-b965-e3f10d928461-kube-api-access-wdlxc\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.621171 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.621014    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-ca\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.621171 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.621066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.621171 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.621103    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d6d6fd-0047-4fb7-b965-e3f10d928461-additional-service-ca\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.621171 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.621127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-hub\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.621171 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.621147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.622303 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.622243    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d6d6fd-0047-4fb7-b965-e3f10d928461-additional-service-ca\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.624444 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.624414    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-hub\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.625313 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.625285    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.626014 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.625992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-ca\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.626107 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.626080    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b5d6d6fd-0047-4fb7-b965-e3f10d928461-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.633126 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.633102    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdlxc\" (UniqueName: \"kubernetes.io/projected/b5d6d6fd-0047-4fb7-b965-e3f10d928461-kube-api-access-wdlxc\") pod \"cluster-proxy-proxy-agent-984bd6987-7rjvv\" (UID: \"b5d6d6fd-0047-4fb7-b965-e3f10d928461\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:12.776857 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:12.776822    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"
Sep 14 07:50:13.055767 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:13.055675    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv"]
Sep 14 07:50:13.059678 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:13.059647    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5d6d6fd_0047_4fb7_b965_e3f10d928461.slice/crio-7f2fec9d54ccd9a3c6d99aaf085708f54a9bbf059eb89b1e7fa73993662494ad WatchSource:0}: Error finding container 7f2fec9d54ccd9a3c6d99aaf085708f54a9bbf059eb89b1e7fa73993662494ad: Status 404 returned error can't find the container with id 7f2fec9d54ccd9a3c6d99aaf085708f54a9bbf059eb89b1e7fa73993662494ad
Sep 14 07:50:13.413528 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:13.413435    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tqc4" event={"ID":"c2f14a15-357d-45a7-84f8-5a4d10b369c4","Type":"ContainerStarted","Data":"c1401449a5f1ec1b9c8d90bdee95081093161a35141e08832f20f613ada6a3b4"}
Sep 14 07:50:13.413528 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:13.413498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tqc4" event={"ID":"c2f14a15-357d-45a7-84f8-5a4d10b369c4","Type":"ContainerStarted","Data":"5cadbbe41f5d208c39e45948f0702691871f68f58441bb69e83c621b02f9eaf5"}
Sep 14 07:50:13.413951 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:13.413551    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:50:13.414891 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:13.414862    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv" event={"ID":"b5d6d6fd-0047-4fb7-b965-e3f10d928461","Type":"ContainerStarted","Data":"7f2fec9d54ccd9a3c6d99aaf085708f54a9bbf059eb89b1e7fa73993662494ad"}
Sep 14 07:50:13.429523 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:13.429468    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9tqc4" podStartSLOduration=33.456754596 podStartE2EDuration="35.429452421s" totalImagesPullingTime="1.972697825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:13.428309485 +0000 UTC m=+69.927702289" watchObservedRunningTime="2026-09-14 07:50:13.429452421 +0000 UTC m=+69.928845199"
Sep 14 07:50:15.425850 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:15.425808    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tc6ts" event={"ID":"60928bfa-f870-48b6-9806-1975c0990803","Type":"ContainerStarted","Data":"b412cefe5b091b82b1377e42869e8770a0b03230144f0b57a64008a409845a00"}
Sep 14 07:50:15.439287 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:15.438996    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tc6ts" podStartSLOduration=66.191738607 podStartE2EDuration="1m10.438975293s" totalImagesPullingTime="4.247236686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:15.438886319 +0000 UTC m=+71.938279110" watchObservedRunningTime="2026-09-14 07:50:15.438975293 +0000 UTC m=+71.938368085"
Sep 14 07:50:18.441164 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:18.441128    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv" event={"ID":"b5d6d6fd-0047-4fb7-b965-e3f10d928461","Type":"ContainerStarted","Data":"d489e00458eeb5734004d62bc3d394671f228aca91ce937e1efcd77a11dfc420"}
Sep 14 07:50:18.441507 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:18.441181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv" event={"ID":"b5d6d6fd-0047-4fb7-b965-e3f10d928461","Type":"ContainerStarted","Data":"9e673addc43862df7babfcf5d8d26025269213b890dfe5010eb2d1df298fb3a3"}
Sep 14 07:50:18.441507 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:18.441193    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv" event={"ID":"b5d6d6fd-0047-4fb7-b965-e3f10d928461","Type":"ContainerStarted","Data":"6ca3c791eb4c25a354892b5f4017f4bd7ed676f039a1f00e1268e894ab8888cc"}
Sep 14 07:50:18.458777 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:18.458737    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-984bd6987-7rjvv" podStartSLOduration=1.811085652 podStartE2EDuration="6.458723952s" totalImagesPullingTime="4.6476383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:18.457520097 +0000 UTC m=+74.956912886" watchObservedRunningTime="2026-09-14 07:50:18.458723952 +0000 UTC m=+74.958116729"
Sep 14 07:50:23.421699 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:23.421668    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9tqc4"
Sep 14 07:50:30.297103 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.296990    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7gjkd"]
Sep 14 07:50:30.302886 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.302866    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.305469 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.305443    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 07:50:30.305613 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.305568    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcz9h\""
Sep 14 07:50:30.305613 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.305588    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 07:50:30.306448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.306429    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 07:50:30.306598 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.306485    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 07:50:30.306708 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.306536    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 07:50:30.306708 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.306579    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 07:50:30.348680 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348643    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-sys\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.348680 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348679    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-textfile\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.348912 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348705    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94f45\" (UniqueName: \"kubernetes.io/projected/a60e1905-d061-44ce-97f6-a69deaf7bfad-kube-api-access-94f45\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.348912 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-root\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.348912 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a60e1905-d061-44ce-97f6-a69deaf7bfad-metrics-client-ca\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.348912 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348867    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-wtmp\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.348912 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-accelerators-collector-config\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.349109 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348942    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-tls\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.349109 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.348978    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450215 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450225    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-sys\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-textfile\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450269    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94f45\" (UniqueName: \"kubernetes.io/projected/a60e1905-d061-44ce-97f6-a69deaf7bfad-kube-api-access-94f45\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-root\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450306    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a60e1905-d061-44ce-97f6-a69deaf7bfad-metrics-client-ca\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450306    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-sys\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-wtmp\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450403 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450367    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-accelerators-collector-config\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450822 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450417    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-tls\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450822 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450369    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-root\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450822 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450528    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-wtmp\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.450822 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450665    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-textfile\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.451069 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.450912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a60e1905-d061-44ce-97f6-a69deaf7bfad-metrics-client-ca\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.451069 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.451006    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-accelerators-collector-config\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.452473 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.452454    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.452692 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.452670    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a60e1905-d061-44ce-97f6-a69deaf7bfad-node-exporter-tls\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.457164 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.457143    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94f45\" (UniqueName: \"kubernetes.io/projected/a60e1905-d061-44ce-97f6-a69deaf7bfad-kube-api-access-94f45\") pod \"node-exporter-7gjkd\" (UID: \"a60e1905-d061-44ce-97f6-a69deaf7bfad\") " pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.459126 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.459104    2556 patch_prober.go:28] interesting pod/image-registry-5dcf68555b-mswbv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 14 07:50:30.459233 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.459155    2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" podUID="1e8da133-cac2-4e53-88d5-705811ed1c41" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 07:50:30.612813 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:30.612744    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7gjkd"
Sep 14 07:50:30.625503 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:30.625475    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60e1905_d061_44ce_97f6_a69deaf7bfad.slice/crio-878e9c0c37332cea8d0fb65c3bc12e2e0fa86c448f89cb2f710b9a5aab113935 WatchSource:0}: Error finding container 878e9c0c37332cea8d0fb65c3bc12e2e0fa86c448f89cb2f710b9a5aab113935: Status 404 returned error can't find the container with id 878e9c0c37332cea8d0fb65c3bc12e2e0fa86c448f89cb2f710b9a5aab113935
Sep 14 07:50:31.482937 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:31.482901    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gjkd" event={"ID":"a60e1905-d061-44ce-97f6-a69deaf7bfad","Type":"ContainerStarted","Data":"878e9c0c37332cea8d0fb65c3bc12e2e0fa86c448f89cb2f710b9a5aab113935"}
Sep 14 07:50:32.487536 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:32.487498    2556 generic.go:356] "Generic (PLEG): container finished" podID="a60e1905-d061-44ce-97f6-a69deaf7bfad" containerID="d9ea402c3310d41bfd560d35e749cfc43b18c1e22bdf2312e8081e4b9c3a3b0e" exitCode=0
Sep 14 07:50:32.487906 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:32.487588    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gjkd" event={"ID":"a60e1905-d061-44ce-97f6-a69deaf7bfad","Type":"ContainerDied","Data":"d9ea402c3310d41bfd560d35e749cfc43b18c1e22bdf2312e8081e4b9c3a3b0e"}
Sep 14 07:50:33.419195 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:33.419159    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:50:33.494295 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:33.494258    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gjkd" event={"ID":"a60e1905-d061-44ce-97f6-a69deaf7bfad","Type":"ContainerStarted","Data":"bc2cb0f7149ede87c2a1aef8548df2b549b1b525e18421da07ddc4f38de942b8"}
Sep 14 07:50:33.494737 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:33.494317    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gjkd" event={"ID":"a60e1905-d061-44ce-97f6-a69deaf7bfad","Type":"ContainerStarted","Data":"6940691af0787278f5d611778b19491a2a4cdb7edf06ec1b516707b14c7ea5a8"}
Sep 14 07:50:33.512320 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:33.511768    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7gjkd" podStartSLOduration=2.55907574 podStartE2EDuration="3.511747482s" totalImagesPullingTime="892.940935ms" totalInitContainerRuntime="59.730807ms" isStatefulPod=true podCreationTimestamp="2026-09-14 07:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:33.511072016 +0000 UTC m=+90.010464812" watchObservedRunningTime="2026-09-14 07:50:33.511747482 +0000 UTC m=+90.011140262"
Sep 14 07:50:34.572168 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.572134    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8f74c88f-57chb"]
Sep 14 07:50:34.575156 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.575140    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.577845 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.577823    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 14 07:50:34.577956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.577904    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3a2h6anhgbp9p\""
Sep 14 07:50:34.577956 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.577831    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 14 07:50:34.578113 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.577844    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dzw9d\""
Sep 14 07:50:34.578113 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.577856    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 14 07:50:34.578113 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.577824    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 07:50:34.582926 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.582899    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8f74c88f-57chb"]
Sep 14 07:50:34.683934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.683899    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.683934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.683941    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-metrics-server-audit-profiles\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.684138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.683977    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-secret-metrics-server-client-certs\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.684138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.684000    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-audit-log\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.684138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.684019    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-client-ca-bundle\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.684138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.684044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2wm8\" (UniqueName: \"kubernetes.io/projected/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-kube-api-access-p2wm8\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.684138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.684125    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-secret-metrics-server-tls\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785245 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-secret-metrics-server-tls\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785402 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785402 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785290    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-metrics-server-audit-profiles\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785507 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-secret-metrics-server-client-certs\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785507 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785448    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-audit-log\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785507 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-client-ca-bundle\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785703 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2wm8\" (UniqueName: \"kubernetes.io/projected/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-kube-api-access-p2wm8\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.785806 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.785783    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-audit-log\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.786174 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.786151    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.786284 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.786263    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-metrics-server-audit-profiles\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.787792 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.787767    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-secret-metrics-server-client-certs\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.787887 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.787848    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-client-ca-bundle\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.787887 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.787870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-secret-metrics-server-tls\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.792380 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.792359    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2wm8\" (UniqueName: \"kubernetes.io/projected/4546a0bc-bfd5-47f7-a0fb-afa108d9f027-kube-api-access-p2wm8\") pod \"metrics-server-8f74c88f-57chb\" (UID: \"4546a0bc-bfd5-47f7-a0fb-afa108d9f027\") " pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:34.884924 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:34.884835    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:35.156034 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.155921    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8f74c88f-57chb"]
Sep 14 07:50:35.159453 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:35.159415    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4546a0bc_bfd5_47f7_a0fb_afa108d9f027.slice/crio-88b475518307239d6073f9fbc02812faf5d97bb0f2c1c6ea7874b56386c32e9a WatchSource:0}: Error finding container 88b475518307239d6073f9fbc02812faf5d97bb0f2c1c6ea7874b56386c32e9a: Status 404 returned error can't find the container with id 88b475518307239d6073f9fbc02812faf5d97bb0f2c1c6ea7874b56386c32e9a
Sep 14 07:50:35.205523 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.205491    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8569dfdd7f-mpftr"]
Sep 14 07:50:35.209684 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.209666    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.211996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.211949    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 07:50:35.211996 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.211952    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 07:50:35.212175 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.211957    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hvfj5\""
Sep 14 07:50:35.212175 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.212086    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 07:50:35.212175 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.212119    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 07:50:35.212323 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.212310    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 07:50:35.212668 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.212651    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 07:50:35.212733 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.212719    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 07:50:35.216655 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.216630    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 07:50:35.218824 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.218805    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8569dfdd7f-mpftr"]
Sep 14 07:50:35.290066 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290039    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-oauth-config\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.290217 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-console-config\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.290217 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290101    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-serving-cert\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.290217 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290150    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgh6q\" (UniqueName: \"kubernetes.io/projected/7b5af990-7836-40a0-a040-f670cf913039-kube-api-access-dgh6q\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.290217 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290205    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-service-ca\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.290406 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-trusted-ca-bundle\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.290406 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.290323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-oauth-serving-cert\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391367 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391326    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-service-ca\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391367 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391371    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-trusted-ca-bundle\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391627 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-oauth-serving-cert\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391627 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391424    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-oauth-config\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391627 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-console-config\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391627 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391591    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-serving-cert\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.391821 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.391729    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgh6q\" (UniqueName: \"kubernetes.io/projected/7b5af990-7836-40a0-a040-f670cf913039-kube-api-access-dgh6q\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.392175 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.392123    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-service-ca\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.392304 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.392289    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-oauth-serving-cert\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.392473 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.392453    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-trusted-ca-bundle\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.392565 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.392547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-console-config\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.394311 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.394286    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-oauth-config\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.394495 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.394475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-serving-cert\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.399432 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.399409    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgh6q\" (UniqueName: \"kubernetes.io/projected/7b5af990-7836-40a0-a040-f670cf913039-kube-api-access-dgh6q\") pod \"console-8569dfdd7f-mpftr\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") " pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.502579 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.502547    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8f74c88f-57chb" event={"ID":"4546a0bc-bfd5-47f7-a0fb-afa108d9f027","Type":"ContainerStarted","Data":"88b475518307239d6073f9fbc02812faf5d97bb0f2c1c6ea7874b56386c32e9a"}
Sep 14 07:50:35.520756 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.520727    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:35.791585 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:35.791544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8569dfdd7f-mpftr"]
Sep 14 07:50:35.794375 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:50:35.794347    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5af990_7836_40a0_a040_f670cf913039.slice/crio-e0ccab706973030831750ca359aee3d43bfc9a27b40c08ec744de53be04bb6ec WatchSource:0}: Error finding container e0ccab706973030831750ca359aee3d43bfc9a27b40c08ec744de53be04bb6ec: Status 404 returned error can't find the container with id e0ccab706973030831750ca359aee3d43bfc9a27b40c08ec744de53be04bb6ec
Sep 14 07:50:36.195241 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:36.195206    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dcf68555b-mswbv"]
Sep 14 07:50:36.509353 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:36.509267    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8569dfdd7f-mpftr" event={"ID":"7b5af990-7836-40a0-a040-f670cf913039","Type":"ContainerStarted","Data":"e0ccab706973030831750ca359aee3d43bfc9a27b40c08ec744de53be04bb6ec"}
Sep 14 07:50:37.515584 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:37.515547    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8f74c88f-57chb" event={"ID":"4546a0bc-bfd5-47f7-a0fb-afa108d9f027","Type":"ContainerStarted","Data":"7a449d48a749d5e7a766a0734ef26167eb87154ff7256a03fcd7dd46c4cb6dd1"}
Sep 14 07:50:37.532019 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:37.531945    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8f74c88f-57chb" podStartSLOduration=1.999176389 podStartE2EDuration="3.53192854s" totalImagesPullingTime="1.532752151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:37.53098338 +0000 UTC m=+94.030376178" watchObservedRunningTime="2026-09-14 07:50:37.53192854 +0000 UTC m=+94.031321317"
Sep 14 07:50:39.524657 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:39.524617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8569dfdd7f-mpftr" event={"ID":"7b5af990-7836-40a0-a040-f670cf913039","Type":"ContainerStarted","Data":"6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315"}
Sep 14 07:50:39.547081 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:39.547032    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8569dfdd7f-mpftr" podStartSLOduration=1.848850434 podStartE2EDuration="4.547014735s" totalImagesPullingTime="2.698164301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:39.545766237 +0000 UTC m=+96.045159033" watchObservedRunningTime="2026-09-14 07:50:39.547014735 +0000 UTC m=+96.046407513"
Sep 14 07:50:41.402719 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:41.402689    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n5ks4"
Sep 14 07:50:45.521862 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:45.521827    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:45.522267 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:45.521906    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:45.526631 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:45.526609    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:45.549192 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:45.549171    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:50:54.886004 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:54.885931    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:50:54.886471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:50:54.886037    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:51:01.214406 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.214338    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" podUID="1e8da133-cac2-4e53-88d5-705811ed1c41" containerName="registry" containerID="cri-o://274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619" gracePeriod=30
Sep 14 07:51:01.448882 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.448850    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:51:01.600436 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.600343    2556 generic.go:356] "Generic (PLEG): container finished" podID="1e8da133-cac2-4e53-88d5-705811ed1c41" containerID="274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619" exitCode=0
Sep 14 07:51:01.600436 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.600388    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv"
Sep 14 07:51:01.600609 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.600436    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" event={"ID":"1e8da133-cac2-4e53-88d5-705811ed1c41","Type":"ContainerDied","Data":"274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619"}
Sep 14 07:51:01.600609 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.600483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dcf68555b-mswbv" event={"ID":"1e8da133-cac2-4e53-88d5-705811ed1c41","Type":"ContainerDied","Data":"3a391506741f86232cd046a5314e838053de81af5c141b8a7ad7e0ca3cb3fb5a"}
Sep 14 07:51:01.600609 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.600503    2556 scope.go:117] "RemoveContainer" containerID="274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619"
Sep 14 07:51:01.605239 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605219    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605381 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605254    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddwv5\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-kube-api-access-ddwv5\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605381 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605291    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-image-registry-private-configuration\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605381 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605328    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e8da133-cac2-4e53-88d5-705811ed1c41-ca-trust-extracted\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605381 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605373    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-trusted-ca\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605583 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605430    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-bound-sa-token\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605583 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605455    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-installation-pull-secrets\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.605583 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.605484    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-certificates\") pod \"1e8da133-cac2-4e53-88d5-705811ed1c41\" (UID: \"1e8da133-cac2-4e53-88d5-705811ed1c41\") "
Sep 14 07:51:01.606079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.606021    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:01.606079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.606072    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:01.608023 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.607981    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:01.608306 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.608243    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:01.608397 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.608311    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:01.608397 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.608338    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:01.608528 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.608508    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-kube-api-access-ddwv5" (OuterVolumeSpecName: "kube-api-access-ddwv5") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "kube-api-access-ddwv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:01.609815 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.609796    2556 scope.go:117] "RemoveContainer" containerID="274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619"
Sep 14 07:51:01.610166 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:51:01.610142    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619\": container with ID starting with 274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619 not found: ID does not exist" containerID="274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619"
Sep 14 07:51:01.610230 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.610172    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619"} err="failed to get container status \"274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619\": rpc error: code = NotFound desc = could not find container \"274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619\": container with ID starting with 274980a8bd96e198f505efbe00bcff543aff13c7395cae6a01bd9a75959e1619 not found: ID does not exist"
Sep 14 07:51:01.616917 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.616891    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8da133-cac2-4e53-88d5-705811ed1c41-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1e8da133-cac2-4e53-88d5-705811ed1c41" (UID: "1e8da133-cac2-4e53-88d5-705811ed1c41"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:51:01.706599 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706566    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706599 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706598    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddwv5\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-kube-api-access-ddwv5\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706763 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706611    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-image-registry-private-configuration\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706763 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706621    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e8da133-cac2-4e53-88d5-705811ed1c41-ca-trust-extracted\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706763 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706630    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-trusted-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706763 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706637    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e8da133-cac2-4e53-88d5-705811ed1c41-bound-sa-token\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706763 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706645    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e8da133-cac2-4e53-88d5-705811ed1c41-installation-pull-secrets\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.706763 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.706654    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e8da133-cac2-4e53-88d5-705811ed1c41-registry-certificates\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:51:01.920563 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.920535    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dcf68555b-mswbv"]
Sep 14 07:51:01.924388 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:01.924364    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5dcf68555b-mswbv"]
Sep 14 07:51:02.016994 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:02.016948    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e8da133-cac2-4e53-88d5-705811ed1c41" path="/var/lib/kubelet/pods/1e8da133-cac2-4e53-88d5-705811ed1c41/volumes"
Sep 14 07:51:10.634314 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:10.634234    2556 generic.go:356] "Generic (PLEG): container finished" podID="61186fd6-9d8f-43b2-967c-9c1752ced815" containerID="1d932bb1a8f556ed19139dee0a806455a5795332596baae7b7cd9999d3b0b1c8" exitCode=0
Sep 14 07:51:10.634314 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:10.634280    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58" event={"ID":"61186fd6-9d8f-43b2-967c-9c1752ced815","Type":"ContainerDied","Data":"1d932bb1a8f556ed19139dee0a806455a5795332596baae7b7cd9999d3b0b1c8"}
Sep 14 07:51:10.634678 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:10.634534    2556 scope.go:117] "RemoveContainer" containerID="1d932bb1a8f556ed19139dee0a806455a5795332596baae7b7cd9999d3b0b1c8"
Sep 14 07:51:11.639552 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:11.639515    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wsm58" event={"ID":"61186fd6-9d8f-43b2-967c-9c1752ced815","Type":"ContainerStarted","Data":"7b583918126f30f23e03e4218b741745b46eae8f6c2ca3b0e3a26eb2aacb8e44"}
Sep 14 07:51:14.890519 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:14.890487    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:51:14.894267 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:14.894245    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8f74c88f-57chb"
Sep 14 07:51:57.814543 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.814463    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c94fc5466-cvbgv"]
Sep 14 07:51:57.814931 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.814710    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8da133-cac2-4e53-88d5-705811ed1c41" containerName="registry"
Sep 14 07:51:57.814931 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.814720    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8da133-cac2-4e53-88d5-705811ed1c41" containerName="registry"
Sep 14 07:51:57.814931 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.814769    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8da133-cac2-4e53-88d5-705811ed1c41" containerName="registry"
Sep 14 07:51:57.817687 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.817670    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.830354 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.830329    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c94fc5466-cvbgv"]
Sep 14 07:51:57.898376 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-config\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.898376 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898377    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-service-ca\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.898579 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898464    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-kube-api-access-tb9zp\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.898579 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-oauth-serving-cert\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.898579 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898551    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-serving-cert\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.898579 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898574    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-oauth-config\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.898704 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.898590    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-trusted-ca-bundle\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999630 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-kube-api-access-tb9zp\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999630 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999636    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-oauth-serving-cert\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999659    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-serving-cert\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-oauth-config\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-trusted-ca-bundle\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999725    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-config\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:57.999894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:57.999748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-service-ca\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.000612 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.000586    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-service-ca\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.000724 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.000586    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-oauth-serving-cert\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.000776 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.000713    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-config\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.000776 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.000713    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-trusted-ca-bundle\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.002266 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.002241    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-oauth-config\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.002363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.002320    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-serving-cert\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.007076 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.007054    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-kube-api-access-tb9zp\") pod \"console-7c94fc5466-cvbgv\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") " pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.125986 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.125881    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:51:58.401058 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.400957    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c94fc5466-cvbgv"]
Sep 14 07:51:58.404159 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:51:58.404126    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5918fe_0146_49e8_8d87_fcdc84cbee8a.slice/crio-6d493a07c0523a2a5cdcaa6a75f2061e3f6e155cea1fe15086981f949188733d WatchSource:0}: Error finding container 6d493a07c0523a2a5cdcaa6a75f2061e3f6e155cea1fe15086981f949188733d: Status 404 returned error can't find the container with id 6d493a07c0523a2a5cdcaa6a75f2061e3f6e155cea1fe15086981f949188733d
Sep 14 07:51:58.802688 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.802646    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94fc5466-cvbgv" event={"ID":"7c5918fe-0146-49e8-8d87-fcdc84cbee8a","Type":"ContainerStarted","Data":"30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d"}
Sep 14 07:51:58.802852 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.802708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94fc5466-cvbgv" event={"ID":"7c5918fe-0146-49e8-8d87-fcdc84cbee8a","Type":"ContainerStarted","Data":"6d493a07c0523a2a5cdcaa6a75f2061e3f6e155cea1fe15086981f949188733d"}
Sep 14 07:51:58.819022 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:51:58.818937    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c94fc5466-cvbgv" podStartSLOduration=1.818919992 podStartE2EDuration="1.818919992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:51:58.817932402 +0000 UTC m=+175.317325298" watchObservedRunningTime="2026-09-14 07:51:58.818919992 +0000 UTC m=+175.318312769"
Sep 14 07:52:06.087323 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.087285    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c94fc5466-cvbgv"]
Sep 14 07:52:06.118138 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.118102    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65bf895c7c-n8vnn"]
Sep 14 07:52:06.122499 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.122473    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.133766 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.133741    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65bf895c7c-n8vnn"]
Sep 14 07:52:06.272934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.272895    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-oauth-config\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.272934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.272932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-serving-cert\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.273149 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.272954    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-trusted-ca-bundle\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.273149 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.273092    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/f519d385-49d3-42d3-9340-9c0efe3142e0-kube-api-access-4lt5n\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.273149 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.273130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-oauth-serving-cert\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.273242 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.273157    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-service-ca\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.273242 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.273188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-console-config\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374400    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/f519d385-49d3-42d3-9340-9c0efe3142e0-kube-api-access-4lt5n\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374456    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-oauth-serving-cert\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374668 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374504    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-service-ca\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374668 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374535    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-console-config\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374668 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-oauth-config\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374668 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374574    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-serving-cert\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.374668 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.374590    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-trusted-ca-bundle\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.375271 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.375245    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-console-config\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.375378 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.375244    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-oauth-serving-cert\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.375378 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.375244    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-service-ca\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.375499 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.375479    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-trusted-ca-bundle\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.377560 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.377533    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-serving-cert\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.377665 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.377645    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-oauth-config\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.382099 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.382077    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/f519d385-49d3-42d3-9340-9c0efe3142e0-kube-api-access-4lt5n\") pod \"console-65bf895c7c-n8vnn\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") " pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.430990 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.430933    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:06.701462 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.701435    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65bf895c7c-n8vnn"]
Sep 14 07:52:06.705206 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:52:06.705174    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf519d385_49d3_42d3_9340_9c0efe3142e0.slice/crio-c1e11791952987d92b1cffa73d747ed42c5821592129b4f3abaee1a6d6a884a5 WatchSource:0}: Error finding container c1e11791952987d92b1cffa73d747ed42c5821592129b4f3abaee1a6d6a884a5: Status 404 returned error can't find the container with id c1e11791952987d92b1cffa73d747ed42c5821592129b4f3abaee1a6d6a884a5
Sep 14 07:52:06.836826 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.836787    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65bf895c7c-n8vnn" event={"ID":"f519d385-49d3-42d3-9340-9c0efe3142e0","Type":"ContainerStarted","Data":"515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901"}
Sep 14 07:52:06.837048 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.836876    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65bf895c7c-n8vnn" event={"ID":"f519d385-49d3-42d3-9340-9c0efe3142e0","Type":"ContainerStarted","Data":"c1e11791952987d92b1cffa73d747ed42c5821592129b4f3abaee1a6d6a884a5"}
Sep 14 07:52:06.854723 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:06.854664    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65bf895c7c-n8vnn" podStartSLOduration=0.854648365 podStartE2EDuration="854.648365ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:52:06.853526646 +0000 UTC m=+183.352919424" watchObservedRunningTime="2026-09-14 07:52:06.854648365 +0000 UTC m=+183.354041143"
Sep 14 07:52:08.126582 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:08.126544    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:52:16.431214 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:16.431164    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:16.431797 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:16.431241    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:16.436043 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:16.436021    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:16.875926 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:16.875838    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:52:16.920749 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:16.920720    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8569dfdd7f-mpftr"]
Sep 14 07:52:31.105879 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.105840    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c94fc5466-cvbgv" podUID="7c5918fe-0146-49e8-8d87-fcdc84cbee8a" containerName="console" containerID="cri-o://30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d" gracePeriod=15
Sep 14 07:52:31.340909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.340880    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:52:31.476200 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476169    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-serving-cert\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476238    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-service-ca\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476275    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-oauth-config\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476294    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-trusted-ca-bundle\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476321    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-kube-api-access-tb9zp\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476382 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476364    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-oauth-serving-cert\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476647 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476393    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-config\") pod \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\" (UID: \"7c5918fe-0146-49e8-8d87-fcdc84cbee8a\") "
Sep 14 07:52:31.476760 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476733    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-service-ca" (OuterVolumeSpecName: "service-ca") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:31.476872 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476841    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-config" (OuterVolumeSpecName: "console-config") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:31.477015 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476920    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:31.477015 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.476928    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:31.478498 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.478473    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:52:31.478852 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.478827    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:52:31.478852 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.478836    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-kube-api-access-tb9zp" (OuterVolumeSpecName: "kube-api-access-tb9zp") pod "7c5918fe-0146-49e8-8d87-fcdc84cbee8a" (UID: "7c5918fe-0146-49e8-8d87-fcdc84cbee8a"). InnerVolumeSpecName "kube-api-access-tb9zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:52:31.577489 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577440    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.577489 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577483    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.577489 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577493    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.577489 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577505    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-kube-api-access-tb9zp\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.577743 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577514    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.577743 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577522    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.577743 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.577530    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c5918fe-0146-49e8-8d87-fcdc84cbee8a-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:31.924838 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.924806    2556 generic.go:356] "Generic (PLEG): container finished" podID="7c5918fe-0146-49e8-8d87-fcdc84cbee8a" containerID="30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d" exitCode=0
Sep 14 07:52:31.925019 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.924857    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94fc5466-cvbgv"
Sep 14 07:52:31.925019 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.924903    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94fc5466-cvbgv" event={"ID":"7c5918fe-0146-49e8-8d87-fcdc84cbee8a","Type":"ContainerDied","Data":"30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d"}
Sep 14 07:52:31.925019 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.924977    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94fc5466-cvbgv" event={"ID":"7c5918fe-0146-49e8-8d87-fcdc84cbee8a","Type":"ContainerDied","Data":"6d493a07c0523a2a5cdcaa6a75f2061e3f6e155cea1fe15086981f949188733d"}
Sep 14 07:52:31.925019 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.925002    2556 scope.go:117] "RemoveContainer" containerID="30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d"
Sep 14 07:52:31.933774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.933759    2556 scope.go:117] "RemoveContainer" containerID="30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d"
Sep 14 07:52:31.934040 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:52:31.934019    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d\": container with ID starting with 30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d not found: ID does not exist" containerID="30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d"
Sep 14 07:52:31.934133 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.934047    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d"} err="failed to get container status \"30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d\": rpc error: code = NotFound desc = could not find container \"30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d\": container with ID starting with 30dbc2d3ec140402064912c1a5b8a4f9e2eae63a16e73fe9a1e35dc48f65a96d not found: ID does not exist"
Sep 14 07:52:31.944817 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.944796    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c94fc5466-cvbgv"]
Sep 14 07:52:31.947229 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:31.947209    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c94fc5466-cvbgv"]
Sep 14 07:52:32.017619 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:32.017586    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c5918fe-0146-49e8-8d87-fcdc84cbee8a" path="/var/lib/kubelet/pods/7c5918fe-0146-49e8-8d87-fcdc84cbee8a/volumes"
Sep 14 07:52:41.939492 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:41.939430    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8569dfdd7f-mpftr" podUID="7b5af990-7836-40a0-a040-f670cf913039" containerName="console" containerID="cri-o://6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315" gracePeriod=15
Sep 14 07:52:42.173584 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.173557    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:52:42.248926 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.248896    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-oauth-config\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.248940    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-oauth-serving-cert\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.248982    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-service-ca\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249004    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-console-config\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249026    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgh6q\" (UniqueName: \"kubernetes.io/projected/7b5af990-7836-40a0-a040-f670cf913039-kube-api-access-dgh6q\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249079 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249049    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-serving-cert\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249319 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249090    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-trusted-ca-bundle\") pod \"7b5af990-7836-40a0-a040-f670cf913039\" (UID: \"7b5af990-7836-40a0-a040-f670cf913039\") "
Sep 14 07:52:42.249512 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249478    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-service-ca" (OuterVolumeSpecName: "service-ca") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:42.249512 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249492    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:42.249703 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249499    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-console-config" (OuterVolumeSpecName: "console-config") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:42.249703 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.249557    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:52:42.251179 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.251155    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:52:42.251267 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.251191    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b5af990-7836-40a0-a040-f670cf913039-kube-api-access-dgh6q" (OuterVolumeSpecName: "kube-api-access-dgh6q") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "kube-api-access-dgh6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:52:42.251466 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.251447    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7b5af990-7836-40a0-a040-f670cf913039" (UID: "7b5af990-7836-40a0-a040-f670cf913039"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:52:42.350052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.349996    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.350052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.350044    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.350052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.350054    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.350052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.350063    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.350052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.350073    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b5af990-7836-40a0-a040-f670cf913039-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.350347 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.350081    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgh6q\" (UniqueName: \"kubernetes.io/projected/7b5af990-7836-40a0-a040-f670cf913039-kube-api-access-dgh6q\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.350347 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.350090    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5af990-7836-40a0-a040-f670cf913039-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:52:42.964249 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.964208    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b5af990-7836-40a0-a040-f670cf913039" containerID="6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315" exitCode=0
Sep 14 07:52:42.964640 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.964253    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8569dfdd7f-mpftr"
Sep 14 07:52:42.964640 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.964292    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8569dfdd7f-mpftr" event={"ID":"7b5af990-7836-40a0-a040-f670cf913039","Type":"ContainerDied","Data":"6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315"}
Sep 14 07:52:42.964640 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.964339    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8569dfdd7f-mpftr" event={"ID":"7b5af990-7836-40a0-a040-f670cf913039","Type":"ContainerDied","Data":"e0ccab706973030831750ca359aee3d43bfc9a27b40c08ec744de53be04bb6ec"}
Sep 14 07:52:42.964640 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.964360    2556 scope.go:117] "RemoveContainer" containerID="6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315"
Sep 14 07:52:42.972454 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.972432    2556 scope.go:117] "RemoveContainer" containerID="6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315"
Sep 14 07:52:42.972724 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:52:42.972706    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315\": container with ID starting with 6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315 not found: ID does not exist" containerID="6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315"
Sep 14 07:52:42.972773 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.972734    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315"} err="failed to get container status \"6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315\": rpc error: code = NotFound desc = could not find container \"6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315\": container with ID starting with 6c28dba4bd55ed53282f00c412f0ee81dc7cf47ba8db5824e0cc66db0b72a315 not found: ID does not exist"
Sep 14 07:52:42.982751 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.982728    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8569dfdd7f-mpftr"]
Sep 14 07:52:42.986448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:42.986427    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8569dfdd7f-mpftr"]
Sep 14 07:52:44.017163 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:52:44.017128    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b5af990-7836-40a0-a040-f670cf913039" path="/var/lib/kubelet/pods/7b5af990-7836-40a0-a040-f670cf913039/volumes"
Sep 14 07:53:31.610126 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610056    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"]
Sep 14 07:53:31.610531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610325    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c5918fe-0146-49e8-8d87-fcdc84cbee8a" containerName="console"
Sep 14 07:53:31.610531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610337    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c5918fe-0146-49e8-8d87-fcdc84cbee8a" containerName="console"
Sep 14 07:53:31.610531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610349    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b5af990-7836-40a0-a040-f670cf913039" containerName="console"
Sep 14 07:53:31.610531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610355    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b5af990-7836-40a0-a040-f670cf913039" containerName="console"
Sep 14 07:53:31.610531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610397    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c5918fe-0146-49e8-8d87-fcdc84cbee8a" containerName="console"
Sep 14 07:53:31.610531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.610404    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b5af990-7836-40a0-a040-f670cf913039" containerName="console"
Sep 14 07:53:31.613375 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.613360    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.615444 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.615416    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:53:31.615559 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.615417    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:53:31.616143 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.616127    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9nkq4\""
Sep 14 07:53:31.620589 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.620566    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"]
Sep 14 07:53:31.688745 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.688710    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.688919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.688757    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-src2b\" (UniqueName: \"kubernetes.io/projected/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-kube-api-access-src2b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.688919 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.688810    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.790023 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.789993    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.790177 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.790041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-src2b\" (UniqueName: \"kubernetes.io/projected/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-kube-api-access-src2b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.790177 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.790068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.790371 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.790351    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.790430 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.790377    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.797943 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.797914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-src2b\" (UniqueName: \"kubernetes.io/projected/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-kube-api-access-src2b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:31.922402 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:31.922373    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:32.188787 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:32.188755    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"]
Sep 14 07:53:32.190893 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:53:32.190863    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afbbd38_6f71_42e2_ab47_0cc7ff8dc3dd.slice/crio-08d7126e555a027b3c62e7b2bd85140a6d29e7283f8e5418534fbd617f379ad6 WatchSource:0}: Error finding container 08d7126e555a027b3c62e7b2bd85140a6d29e7283f8e5418534fbd617f379ad6: Status 404 returned error can't find the container with id 08d7126e555a027b3c62e7b2bd85140a6d29e7283f8e5418534fbd617f379ad6
Sep 14 07:53:33.133073 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:33.133040    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd" event={"ID":"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd","Type":"ContainerStarted","Data":"08d7126e555a027b3c62e7b2bd85140a6d29e7283f8e5418534fbd617f379ad6"}
Sep 14 07:53:39.158192 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:39.158152    2556 generic.go:356] "Generic (PLEG): container finished" podID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerID="caacb38769140d6ea12ad21064fd523c5afd976121e2e5f7d45ad1d4534ea84e" exitCode=0
Sep 14 07:53:39.158583 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:39.158230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd" event={"ID":"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd","Type":"ContainerDied","Data":"caacb38769140d6ea12ad21064fd523c5afd976121e2e5f7d45ad1d4534ea84e"}
Sep 14 07:53:43.175448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:43.175416    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd" event={"ID":"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd","Type":"ContainerStarted","Data":"fd18cf8b6d90b3a6115cb6092479ca5dcae1373c489cd6e5a1d4aa39d4131518"}
Sep 14 07:53:44.180736 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:44.180699    2556 generic.go:356] "Generic (PLEG): container finished" podID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerID="fd18cf8b6d90b3a6115cb6092479ca5dcae1373c489cd6e5a1d4aa39d4131518" exitCode=0
Sep 14 07:53:44.181146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:44.180786    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd" event={"ID":"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd","Type":"ContainerDied","Data":"fd18cf8b6d90b3a6115cb6092479ca5dcae1373c489cd6e5a1d4aa39d4131518"}
Sep 14 07:53:51.208789 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:51.208749    2556 generic.go:356] "Generic (PLEG): container finished" podID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerID="a66ad65d207c87ccd83287fd4bc3649b763bc378153a402427ea3be082b757ee" exitCode=0
Sep 14 07:53:51.209186 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:51.208831    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd" event={"ID":"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd","Type":"ContainerDied","Data":"a66ad65d207c87ccd83287fd4bc3649b763bc378153a402427ea3be082b757ee"}
Sep 14 07:53:52.353643 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.353616    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:52.460486 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.460449    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-util\") pod \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") "
Sep 14 07:53:52.460659 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.460502    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-bundle\") pod \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") "
Sep 14 07:53:52.460659 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.460532    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-src2b\" (UniqueName: \"kubernetes.io/projected/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-kube-api-access-src2b\") pod \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\" (UID: \"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd\") "
Sep 14 07:53:52.461305 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.461265    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-bundle" (OuterVolumeSpecName: "bundle") pod "1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" (UID: "1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:52.462726 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.462700    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-kube-api-access-src2b" (OuterVolumeSpecName: "kube-api-access-src2b") pod "1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" (UID: "1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd"). InnerVolumeSpecName "kube-api-access-src2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:52.466026 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.465977    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-util" (OuterVolumeSpecName: "util") pod "1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" (UID: "1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:52.561845 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.561812    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:53:52.561845 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.561838    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:53:52.561845 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:52.561848    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-src2b\" (UniqueName: \"kubernetes.io/projected/1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd-kube-api-access-src2b\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.218343 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:53.218313    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd"
Sep 14 07:53:53.218504 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:53.218312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ds6sd" event={"ID":"1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd","Type":"ContainerDied","Data":"08d7126e555a027b3c62e7b2bd85140a6d29e7283f8e5418534fbd617f379ad6"}
Sep 14 07:53:53.218504 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:53.218430    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08d7126e555a027b3c62e7b2bd85140a6d29e7283f8e5418534fbd617f379ad6"
Sep 14 07:53:59.557623 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557587    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"]
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557837    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="extract"
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557848    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="extract"
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557857    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="util"
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557863    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="util"
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557869    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="pull"
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557874    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="pull"
Sep 14 07:53:59.558044 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.557914    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1afbbd38-6f71-42e2-ab47-0cc7ff8dc3dd" containerName="extract"
Sep 14 07:53:59.560648 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.560632    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.563063 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.563041    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:53:59.563141 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.563081    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-nfzqq\""
Sep 14 07:53:59.563141 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.563102    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 14 07:53:59.574527 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.574501    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"]
Sep 14 07:53:59.710686 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.710651    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4prv\" (UniqueName: \"kubernetes.io/projected/109f7fad-e5d5-46af-818a-35db0938b238-kube-api-access-j4prv\") pod \"cert-manager-operator-controller-manager-55c444bf86-qd6np\" (UID: \"109f7fad-e5d5-46af-818a-35db0938b238\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.710686 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.710692    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/109f7fad-e5d5-46af-818a-35db0938b238-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qd6np\" (UID: \"109f7fad-e5d5-46af-818a-35db0938b238\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.811543 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.811466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4prv\" (UniqueName: \"kubernetes.io/projected/109f7fad-e5d5-46af-818a-35db0938b238-kube-api-access-j4prv\") pod \"cert-manager-operator-controller-manager-55c444bf86-qd6np\" (UID: \"109f7fad-e5d5-46af-818a-35db0938b238\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.811543 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.811507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/109f7fad-e5d5-46af-818a-35db0938b238-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qd6np\" (UID: \"109f7fad-e5d5-46af-818a-35db0938b238\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.811844 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.811828    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/109f7fad-e5d5-46af-818a-35db0938b238-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qd6np\" (UID: \"109f7fad-e5d5-46af-818a-35db0938b238\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.819818 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.819797    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4prv\" (UniqueName: \"kubernetes.io/projected/109f7fad-e5d5-46af-818a-35db0938b238-kube-api-access-j4prv\") pod \"cert-manager-operator-controller-manager-55c444bf86-qd6np\" (UID: \"109f7fad-e5d5-46af-818a-35db0938b238\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:53:59.869061 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:53:59.869029    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"
Sep 14 07:54:00.146747 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:00.146723    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np"]
Sep 14 07:54:00.148768 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:54:00.148741    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109f7fad_e5d5_46af_818a_35db0938b238.slice/crio-b6de8440c234db24e4ecc3e64283e918a2f3dc06869cf4f752fd1f0857301287 WatchSource:0}: Error finding container b6de8440c234db24e4ecc3e64283e918a2f3dc06869cf4f752fd1f0857301287: Status 404 returned error can't find the container with id b6de8440c234db24e4ecc3e64283e918a2f3dc06869cf4f752fd1f0857301287
Sep 14 07:54:00.248423 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:00.248387    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np" event={"ID":"109f7fad-e5d5-46af-818a-35db0938b238","Type":"ContainerStarted","Data":"b6de8440c234db24e4ecc3e64283e918a2f3dc06869cf4f752fd1f0857301287"}
Sep 14 07:54:02.259419 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:02.259333    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np" event={"ID":"109f7fad-e5d5-46af-818a-35db0938b238","Type":"ContainerStarted","Data":"bf1aa869ff748d7b436e7ff4e5ba26780d3dcc70f08d44c7bb5ae8422bd9d27e"}
Sep 14 07:54:02.280940 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:02.280893    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qd6np" podStartSLOduration=1.503006685 podStartE2EDuration="3.280877753s" totalImagesPullingTime="1.777871068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:02.27917153 +0000 UTC m=+298.778564340" watchObservedRunningTime="2026-09-14 07:54:02.280877753 +0000 UTC m=+298.780270530"
Sep 14 07:54:03.909501 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:03.909472    2556 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 07:54:09.411143 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.411108    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p7hlz"]
Sep 14 07:54:09.414381 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.414365    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.416464 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.416438    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 07:54:09.416592 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.416508    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fdxjl\""
Sep 14 07:54:09.417206 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.417189    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 07:54:09.421205 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.420856    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p7hlz"]
Sep 14 07:54:09.582727 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.582685    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26ccf41a-098f-4151-a29e-cd4c8b2eec94-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p7hlz\" (UID: \"26ccf41a-098f-4151-a29e-cd4c8b2eec94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.582893 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.582740    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzxq6\" (UniqueName: \"kubernetes.io/projected/26ccf41a-098f-4151-a29e-cd4c8b2eec94-kube-api-access-bzxq6\") pod \"cert-manager-webhook-6486ffd997-p7hlz\" (UID: \"26ccf41a-098f-4151-a29e-cd4c8b2eec94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.683865 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.683831    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26ccf41a-098f-4151-a29e-cd4c8b2eec94-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p7hlz\" (UID: \"26ccf41a-098f-4151-a29e-cd4c8b2eec94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.684031 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.683880    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzxq6\" (UniqueName: \"kubernetes.io/projected/26ccf41a-098f-4151-a29e-cd4c8b2eec94-kube-api-access-bzxq6\") pod \"cert-manager-webhook-6486ffd997-p7hlz\" (UID: \"26ccf41a-098f-4151-a29e-cd4c8b2eec94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.691420 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.691387    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26ccf41a-098f-4151-a29e-cd4c8b2eec94-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p7hlz\" (UID: \"26ccf41a-098f-4151-a29e-cd4c8b2eec94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.691683 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.691662    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzxq6\" (UniqueName: \"kubernetes.io/projected/26ccf41a-098f-4151-a29e-cd4c8b2eec94-kube-api-access-bzxq6\") pod \"cert-manager-webhook-6486ffd997-p7hlz\" (UID: \"26ccf41a-098f-4151-a29e-cd4c8b2eec94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.723363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.723338    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:09.989550 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.989474    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p7hlz"]
Sep 14 07:54:09.992695 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:54:09.992671    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26ccf41a_098f_4151_a29e_cd4c8b2eec94.slice/crio-de9945cadb42cad25c1fc23386a83fa2eb1e1d1f1d66db173d51f6e9b7514f86 WatchSource:0}: Error finding container de9945cadb42cad25c1fc23386a83fa2eb1e1d1f1d66db173d51f6e9b7514f86: Status 404 returned error can't find the container with id de9945cadb42cad25c1fc23386a83fa2eb1e1d1f1d66db173d51f6e9b7514f86
Sep 14 07:54:09.995080 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:09.995064    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:54:10.290468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:10.290385    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz" event={"ID":"26ccf41a-098f-4151-a29e-cd4c8b2eec94","Type":"ContainerStarted","Data":"de9945cadb42cad25c1fc23386a83fa2eb1e1d1f1d66db173d51f6e9b7514f86"}
Sep 14 07:54:13.304421 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.304383    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz" event={"ID":"26ccf41a-098f-4151-a29e-cd4c8b2eec94","Type":"ContainerStarted","Data":"ee98982247296b5b0aad0bf828123e6aec2eb7c9a07aaacc6cbcd89b7d0e34fa"}
Sep 14 07:54:13.304783 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.304445    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:13.309551 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.309525    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xpx8q"]
Sep 14 07:54:13.312601 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.312584    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.314498 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.314479    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9hrv2\""
Sep 14 07:54:13.319082 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.319063    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xpx8q"]
Sep 14 07:54:13.321810 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.321762    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz" podStartSLOduration=1.592848807 podStartE2EDuration="4.321746733s" totalImagesPullingTime="2.728897926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:13.320843018 +0000 UTC m=+309.820235798" watchObservedRunningTime="2026-09-14 07:54:13.321746733 +0000 UTC m=+309.821139513"
Sep 14 07:54:13.409559 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.409527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbc28\" (UniqueName: \"kubernetes.io/projected/c328db60-2497-4df7-b5d8-386fcf376f96-kube-api-access-tbc28\") pod \"cert-manager-55b4d954d5-xpx8q\" (UID: \"c328db60-2497-4df7-b5d8-386fcf376f96\") " pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.409703 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.409568    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c328db60-2497-4df7-b5d8-386fcf376f96-bound-sa-token\") pod \"cert-manager-55b4d954d5-xpx8q\" (UID: \"c328db60-2497-4df7-b5d8-386fcf376f96\") " pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.510906 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.510869    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbc28\" (UniqueName: \"kubernetes.io/projected/c328db60-2497-4df7-b5d8-386fcf376f96-kube-api-access-tbc28\") pod \"cert-manager-55b4d954d5-xpx8q\" (UID: \"c328db60-2497-4df7-b5d8-386fcf376f96\") " pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.511084 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.510918    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c328db60-2497-4df7-b5d8-386fcf376f96-bound-sa-token\") pod \"cert-manager-55b4d954d5-xpx8q\" (UID: \"c328db60-2497-4df7-b5d8-386fcf376f96\") " pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.518583 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.518554    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c328db60-2497-4df7-b5d8-386fcf376f96-bound-sa-token\") pod \"cert-manager-55b4d954d5-xpx8q\" (UID: \"c328db60-2497-4df7-b5d8-386fcf376f96\") " pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.518718 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.518700    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbc28\" (UniqueName: \"kubernetes.io/projected/c328db60-2497-4df7-b5d8-386fcf376f96-kube-api-access-tbc28\") pod \"cert-manager-55b4d954d5-xpx8q\" (UID: \"c328db60-2497-4df7-b5d8-386fcf376f96\") " pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.622175 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.622099    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xpx8q"
Sep 14 07:54:13.891214 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:13.890982    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xpx8q"]
Sep 14 07:54:13.893465 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:54:13.893440    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc328db60_2497_4df7_b5d8_386fcf376f96.slice/crio-b9eeae704d6befc6252310eb8838485974ebef199148b8f26678a6def28c6232 WatchSource:0}: Error finding container b9eeae704d6befc6252310eb8838485974ebef199148b8f26678a6def28c6232: Status 404 returned error can't find the container with id b9eeae704d6befc6252310eb8838485974ebef199148b8f26678a6def28c6232
Sep 14 07:54:14.310328 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:14.310291    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xpx8q" event={"ID":"c328db60-2497-4df7-b5d8-386fcf376f96","Type":"ContainerStarted","Data":"7969c44698e082b7567aa6fdb2432398572e1dd849b5ec5aef2ad81050ef63a3"}
Sep 14 07:54:14.310723 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:14.310339    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xpx8q" event={"ID":"c328db60-2497-4df7-b5d8-386fcf376f96","Type":"ContainerStarted","Data":"b9eeae704d6befc6252310eb8838485974ebef199148b8f26678a6def28c6232"}
Sep 14 07:54:14.326207 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:14.326158    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xpx8q" podStartSLOduration=1.326143667 podStartE2EDuration="1.326143667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:14.324543498 +0000 UTC m=+310.823936275" watchObservedRunningTime="2026-09-14 07:54:14.326143667 +0000 UTC m=+310.825536445"
Sep 14 07:54:19.312518 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:19.312480    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-p7hlz"
Sep 14 07:54:52.854117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.854078    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"]
Sep 14 07:54:52.858394 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.858376    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:52.860488 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.860470    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:54:52.860559 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.860499    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9nkq4\""
Sep 14 07:54:52.861086 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.861071    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:54:52.864424 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.864402    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"]
Sep 14 07:54:52.997394 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.997359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr8mh\" (UniqueName: \"kubernetes.io/projected/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-kube-api-access-cr8mh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:52.997552 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.997410    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:52.997552 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:52.997470    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.098257 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.098215    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr8mh\" (UniqueName: \"kubernetes.io/projected/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-kube-api-access-cr8mh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.098443 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.098279    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.098443 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.098330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.098672 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.098650    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.098751 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.098693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.105850 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.105798    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr8mh\" (UniqueName: \"kubernetes.io/projected/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-kube-api-access-cr8mh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.168397 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.168359    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:53.441028 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.441002    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"]
Sep 14 07:54:53.443530 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:54:53.443506    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906cdcb2_8f51_4a2f_8918_d3400fa8ddf5.slice/crio-66123777c3c83e4148be2c804c8e3e7a1db258b015106fd6ee67b644d0e9b97f WatchSource:0}: Error finding container 66123777c3c83e4148be2c804c8e3e7a1db258b015106fd6ee67b644d0e9b97f: Status 404 returned error can't find the container with id 66123777c3c83e4148be2c804c8e3e7a1db258b015106fd6ee67b644d0e9b97f
Sep 14 07:54:53.460250 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:53.460221    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp" event={"ID":"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5","Type":"ContainerStarted","Data":"66123777c3c83e4148be2c804c8e3e7a1db258b015106fd6ee67b644d0e9b97f"}
Sep 14 07:54:54.465592 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:54.465555    2556 generic.go:356] "Generic (PLEG): container finished" podID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerID="f5a973f85bbb454a37b66a608ccfc1fd103f345e4f11aee0420c14a43d5de3e9" exitCode=0
Sep 14 07:54:54.465958 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:54.465609    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp" event={"ID":"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5","Type":"ContainerDied","Data":"f5a973f85bbb454a37b66a608ccfc1fd103f345e4f11aee0420c14a43d5de3e9"}
Sep 14 07:54:55.472814 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:55.472780    2556 generic.go:356] "Generic (PLEG): container finished" podID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerID="ee8081080b1abcb31a3aff33806e6b7233ca1ee3afeac48102e7bbe29b510868" exitCode=0
Sep 14 07:54:55.473187 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:55.472864    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp" event={"ID":"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5","Type":"ContainerDied","Data":"ee8081080b1abcb31a3aff33806e6b7233ca1ee3afeac48102e7bbe29b510868"}
Sep 14 07:54:56.478958 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:56.478875    2556 generic.go:356] "Generic (PLEG): container finished" podID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerID="21f9ca42c5eb94ae3088c2a0509c9f7a9c81c54e16dca2255b10b804695e186c" exitCode=0
Sep 14 07:54:56.479392 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:56.478979    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp" event={"ID":"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5","Type":"ContainerDied","Data":"21f9ca42c5eb94ae3088c2a0509c9f7a9c81c54e16dca2255b10b804695e186c"}
Sep 14 07:54:57.448033 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.448003    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"]
Sep 14 07:54:57.450892 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.450876    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.453425 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.453403    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:54:57.453425 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.453419    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 07:54:57.453590 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.453444    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 07:54:57.454247 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.454228    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-92nx9\""
Sep 14 07:54:57.454363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.454255    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 07:54:57.454363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.454264    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 07:54:57.463155 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.463133    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"]
Sep 14 07:54:57.537479 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.537444    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnzlq\" (UniqueName: \"kubernetes.io/projected/0ae4c6db-6d53-4212-9e84-0e596d105695-kube-api-access-pnzlq\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.537793 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.537493    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae4c6db-6d53-4212-9e84-0e596d105695-metrics-cert\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.537793 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.537520    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae4c6db-6d53-4212-9e84-0e596d105695-cert\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.537793 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.537590    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae4c6db-6d53-4212-9e84-0e596d105695-manager-config\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.604571 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.604545    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:54:57.638898 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.638862    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnzlq\" (UniqueName: \"kubernetes.io/projected/0ae4c6db-6d53-4212-9e84-0e596d105695-kube-api-access-pnzlq\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.638898 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.638897    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae4c6db-6d53-4212-9e84-0e596d105695-metrics-cert\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.639124 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.638920    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae4c6db-6d53-4212-9e84-0e596d105695-cert\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.639124 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.638974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae4c6db-6d53-4212-9e84-0e596d105695-manager-config\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.639576 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.639552    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae4c6db-6d53-4212-9e84-0e596d105695-manager-config\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.641350 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.641328    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae4c6db-6d53-4212-9e84-0e596d105695-metrics-cert\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.641437 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.641369    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae4c6db-6d53-4212-9e84-0e596d105695-cert\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.649949 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.649925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnzlq\" (UniqueName: \"kubernetes.io/projected/0ae4c6db-6d53-4212-9e84-0e596d105695-kube-api-access-pnzlq\") pod \"lws-controller-manager-7b895dd7c5-qkcs7\" (UID: \"0ae4c6db-6d53-4212-9e84-0e596d105695\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.739909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.739826    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-bundle\") pod \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") "
Sep 14 07:54:57.739909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.739872    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-util\") pod \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") "
Sep 14 07:54:57.740096 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.739929    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr8mh\" (UniqueName: \"kubernetes.io/projected/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-kube-api-access-cr8mh\") pod \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\" (UID: \"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5\") "
Sep 14 07:54:57.740614 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.740582    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-bundle" (OuterVolumeSpecName: "bundle") pod "906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" (UID: "906cdcb2-8f51-4a2f-8918-d3400fa8ddf5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:57.742019 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.741997    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-kube-api-access-cr8mh" (OuterVolumeSpecName: "kube-api-access-cr8mh") pod "906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" (UID: "906cdcb2-8f51-4a2f-8918-d3400fa8ddf5"). InnerVolumeSpecName "kube-api-access-cr8mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:54:57.745398 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.745372    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-util" (OuterVolumeSpecName: "util") pod "906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" (UID: "906cdcb2-8f51-4a2f-8918-d3400fa8ddf5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:57.760156 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.760124    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:54:57.840611 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.840584    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr8mh\" (UniqueName: \"kubernetes.io/projected/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-kube-api-access-cr8mh\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:54:57.840611 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.840609    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:54:57.840774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:57.840619    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906cdcb2-8f51-4a2f-8918-d3400fa8ddf5-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:54:58.035582 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:58.035526    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"]
Sep 14 07:54:58.038055 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:54:58.038029    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae4c6db_6d53_4212_9e84_0e596d105695.slice/crio-2a8ddad0d1bd53cbe428880ab8cc3e1ae8a4c9070c9e392785002f48036ec091 WatchSource:0}: Error finding container 2a8ddad0d1bd53cbe428880ab8cc3e1ae8a4c9070c9e392785002f48036ec091: Status 404 returned error can't find the container with id 2a8ddad0d1bd53cbe428880ab8cc3e1ae8a4c9070c9e392785002f48036ec091
Sep 14 07:54:58.487355 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:58.487320    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7" event={"ID":"0ae4c6db-6d53-4212-9e84-0e596d105695","Type":"ContainerStarted","Data":"2a8ddad0d1bd53cbe428880ab8cc3e1ae8a4c9070c9e392785002f48036ec091"}
Sep 14 07:54:58.489299 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:58.489272    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp" event={"ID":"906cdcb2-8f51-4a2f-8918-d3400fa8ddf5","Type":"ContainerDied","Data":"66123777c3c83e4148be2c804c8e3e7a1db258b015106fd6ee67b644d0e9b97f"}
Sep 14 07:54:58.489417 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:58.489316    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66123777c3c83e4148be2c804c8e3e7a1db258b015106fd6ee67b644d0e9b97f"
Sep 14 07:54:58.489417 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:54:58.489322    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff7vxp"
Sep 14 07:55:00.500139 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:00.500105    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7" event={"ID":"0ae4c6db-6d53-4212-9e84-0e596d105695","Type":"ContainerStarted","Data":"ff7ebd8dae16fc882c35fc95ded7ee27850633ece4d457fcbe7800dbb6aa48cd"}
Sep 14 07:55:00.500512 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:00.500168    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:55:00.516461 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:00.516189    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7" podStartSLOduration=1.229004746 podStartE2EDuration="3.516168895s" totalImagesPullingTime="2.287164149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:00.514893883 +0000 UTC m=+357.014286671" watchObservedRunningTime="2026-09-14 07:55:00.516168895 +0000 UTC m=+357.015561674"
Sep 14 07:55:09.039816 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.039775    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"]
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040081    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="util"
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040094    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="util"
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040109    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="extract"
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040114    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="extract"
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040121    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="pull"
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040127    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="pull"
Sep 14 07:55:09.040332 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.040173    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="906cdcb2-8f51-4a2f-8918-d3400fa8ddf5" containerName="extract"
Sep 14 07:55:09.043221 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.043204    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.045407 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.045384    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:55:09.045532 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.045457    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9nkq4\""
Sep 14 07:55:09.046129 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.046107    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:55:09.051583 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.051561    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"]
Sep 14 07:55:09.133182 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.133145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.133182 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.133181    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2fgf\" (UniqueName: \"kubernetes.io/projected/3d7be046-493a-41f6-876a-ba130d3a6d5d-kube-api-access-r2fgf\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.133393 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.133210    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.234241 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.234203    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.234428 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.234245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fgf\" (UniqueName: \"kubernetes.io/projected/3d7be046-493a-41f6-876a-ba130d3a6d5d-kube-api-access-r2fgf\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.234428 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.234283    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.234566 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.234545    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.234646 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.234626    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.242894 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.242873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fgf\" (UniqueName: \"kubernetes.io/projected/3d7be046-493a-41f6-876a-ba130d3a6d5d-kube-api-access-r2fgf\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.352544 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.352465    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:09.629799 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:09.629773    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"]
Sep 14 07:55:09.632081 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:55:09.632054    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d7be046_493a_41f6_876a_ba130d3a6d5d.slice/crio-30a45737676280f5bf321d38cda004761f23a25158596cd808746f5c3d18883e WatchSource:0}: Error finding container 30a45737676280f5bf321d38cda004761f23a25158596cd808746f5c3d18883e: Status 404 returned error can't find the container with id 30a45737676280f5bf321d38cda004761f23a25158596cd808746f5c3d18883e
Sep 14 07:55:10.024873 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.024840    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"]
Sep 14 07:55:10.027954 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.027936    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.032030 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.032006    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 07:55:10.032141 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.032043    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 07:55:10.032677 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.032660    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 07:55:10.032774 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.032735    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 07:55:10.032940 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.032918    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b4wwj\""
Sep 14 07:55:10.047120 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.047094    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"]
Sep 14 07:55:10.142994 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.142946    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t9qd\" (UniqueName: \"kubernetes.io/projected/308860a3-0a59-4ba6-8c96-d52b471e2810-kube-api-access-9t9qd\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.143166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.143017    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/308860a3-0a59-4ba6-8c96-d52b471e2810-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.143166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.143050    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/308860a3-0a59-4ba6-8c96-d52b471e2810-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.143166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.143082    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/308860a3-0a59-4ba6-8c96-d52b471e2810-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.143166 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.143138    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/308860a3-0a59-4ba6-8c96-d52b471e2810-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.244016 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.243983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/308860a3-0a59-4ba6-8c96-d52b471e2810-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.244174 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.244025    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/308860a3-0a59-4ba6-8c96-d52b471e2810-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.244174 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.244052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/308860a3-0a59-4ba6-8c96-d52b471e2810-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.244174 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.244101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/308860a3-0a59-4ba6-8c96-d52b471e2810-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.244174 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.244147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t9qd\" (UniqueName: \"kubernetes.io/projected/308860a3-0a59-4ba6-8c96-d52b471e2810-kube-api-access-9t9qd\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.244478 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.244432    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/308860a3-0a59-4ba6-8c96-d52b471e2810-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.246290 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.246264    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/308860a3-0a59-4ba6-8c96-d52b471e2810-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.246531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.246514    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/308860a3-0a59-4ba6-8c96-d52b471e2810-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.246659 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.246635    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/308860a3-0a59-4ba6-8c96-d52b471e2810-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.251535 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.251510    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t9qd\" (UniqueName: \"kubernetes.io/projected/308860a3-0a59-4ba6-8c96-d52b471e2810-kube-api-access-9t9qd\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-ttn8h\" (UID: \"308860a3-0a59-4ba6-8c96-d52b471e2810\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.337594 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.337500    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:10.542831 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.542795    2556 generic.go:356] "Generic (PLEG): container finished" podID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerID="20c821b6d542f8c12b994b9f480efec5ec985b4e37ffb8a72e68e16eacdc28e9" exitCode=0
Sep 14 07:55:10.543003 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.542844    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd" event={"ID":"3d7be046-493a-41f6-876a-ba130d3a6d5d","Type":"ContainerDied","Data":"20c821b6d542f8c12b994b9f480efec5ec985b4e37ffb8a72e68e16eacdc28e9"}
Sep 14 07:55:10.543003 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.542897    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd" event={"ID":"3d7be046-493a-41f6-876a-ba130d3a6d5d","Type":"ContainerStarted","Data":"30a45737676280f5bf321d38cda004761f23a25158596cd808746f5c3d18883e"}
Sep 14 07:55:10.612053 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:10.612031    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"]
Sep 14 07:55:10.613787 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:55:10.613759    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308860a3_0a59_4ba6_8c96_d52b471e2810.slice/crio-cfcf0c0b514572d2906e626ee872ec69130b1c3565b87f0eea69614483aae53b WatchSource:0}: Error finding container cfcf0c0b514572d2906e626ee872ec69130b1c3565b87f0eea69614483aae53b: Status 404 returned error can't find the container with id cfcf0c0b514572d2906e626ee872ec69130b1c3565b87f0eea69614483aae53b
Sep 14 07:55:11.506798 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:11.506766    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-qkcs7"
Sep 14 07:55:11.551470 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:11.551417    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h" event={"ID":"308860a3-0a59-4ba6-8c96-d52b471e2810","Type":"ContainerStarted","Data":"cfcf0c0b514572d2906e626ee872ec69130b1c3565b87f0eea69614483aae53b"}
Sep 14 07:55:11.554893 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:11.554361    2556 generic.go:356] "Generic (PLEG): container finished" podID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerID="e2ef6edad9c1a4153ff70f7ba9d199321631904e000b6b76c7f2ff27aa7c9bd0" exitCode=0
Sep 14 07:55:11.554893 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:11.554401    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd" event={"ID":"3d7be046-493a-41f6-876a-ba130d3a6d5d","Type":"ContainerDied","Data":"e2ef6edad9c1a4153ff70f7ba9d199321631904e000b6b76c7f2ff27aa7c9bd0"}
Sep 14 07:55:12.563320 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:12.563278    2556 generic.go:356] "Generic (PLEG): container finished" podID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerID="bbfdb39ddf93cd029422a7729a5fe0fdc3f161650dc6743e428bd14807900df5" exitCode=0
Sep 14 07:55:12.563775 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:12.563361    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd" event={"ID":"3d7be046-493a-41f6-876a-ba130d3a6d5d","Type":"ContainerDied","Data":"bbfdb39ddf93cd029422a7729a5fe0fdc3f161650dc6743e428bd14807900df5"}
Sep 14 07:55:13.770720 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.770693    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:13.871803 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.871720    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-bundle\") pod \"3d7be046-493a-41f6-876a-ba130d3a6d5d\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") "
Sep 14 07:55:13.871803 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.871775    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2fgf\" (UniqueName: \"kubernetes.io/projected/3d7be046-493a-41f6-876a-ba130d3a6d5d-kube-api-access-r2fgf\") pod \"3d7be046-493a-41f6-876a-ba130d3a6d5d\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") "
Sep 14 07:55:13.872028 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.871814    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-util\") pod \"3d7be046-493a-41f6-876a-ba130d3a6d5d\" (UID: \"3d7be046-493a-41f6-876a-ba130d3a6d5d\") "
Sep 14 07:55:13.872471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.872445    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-bundle" (OuterVolumeSpecName: "bundle") pod "3d7be046-493a-41f6-876a-ba130d3a6d5d" (UID: "3d7be046-493a-41f6-876a-ba130d3a6d5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:13.873805 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.873786    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d7be046-493a-41f6-876a-ba130d3a6d5d-kube-api-access-r2fgf" (OuterVolumeSpecName: "kube-api-access-r2fgf") pod "3d7be046-493a-41f6-876a-ba130d3a6d5d" (UID: "3d7be046-493a-41f6-876a-ba130d3a6d5d"). InnerVolumeSpecName "kube-api-access-r2fgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:13.876991 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.876948    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-util" (OuterVolumeSpecName: "util") pod "3d7be046-493a-41f6-876a-ba130d3a6d5d" (UID: "3d7be046-493a-41f6-876a-ba130d3a6d5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:13.972423 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.972387    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:55:13.972423 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.972414    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2fgf\" (UniqueName: \"kubernetes.io/projected/3d7be046-493a-41f6-876a-ba130d3a6d5d-kube-api-access-r2fgf\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:55:13.972423 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:13.972425    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7be046-493a-41f6-876a-ba130d3a6d5d-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:55:14.574169 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:14.574142    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd"
Sep 14 07:55:14.574344 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:14.574170    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0wj4hd" event={"ID":"3d7be046-493a-41f6-876a-ba130d3a6d5d","Type":"ContainerDied","Data":"30a45737676280f5bf321d38cda004761f23a25158596cd808746f5c3d18883e"}
Sep 14 07:55:14.574344 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:14.574208    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30a45737676280f5bf321d38cda004761f23a25158596cd808746f5c3d18883e"
Sep 14 07:55:14.575843 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:14.575819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h" event={"ID":"308860a3-0a59-4ba6-8c96-d52b471e2810","Type":"ContainerStarted","Data":"1faab7ea9e85930f198b2e81fb35e5b46b29ed24c66186854e2b32aaf6d3515a"}
Sep 14 07:55:15.581519 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:15.581484    2556 generic.go:356] "Generic (PLEG): container finished" podID="308860a3-0a59-4ba6-8c96-d52b471e2810" containerID="1faab7ea9e85930f198b2e81fb35e5b46b29ed24c66186854e2b32aaf6d3515a" exitCode=0
Sep 14 07:55:15.582018 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:15.581557    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h" event={"ID":"308860a3-0a59-4ba6-8c96-d52b471e2810","Type":"ContainerDied","Data":"1faab7ea9e85930f198b2e81fb35e5b46b29ed24c66186854e2b32aaf6d3515a"}
Sep 14 07:55:16.587715 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:16.587674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h" event={"ID":"308860a3-0a59-4ba6-8c96-d52b471e2810","Type":"ContainerStarted","Data":"3f048623bdc570abbd10c3475b86239ce484c1b3ece784c2ebc74df8e6a71de5"}
Sep 14 07:55:16.588114 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:16.587816    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:16.608538 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:16.608492    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h" podStartSLOduration=3.033355596 podStartE2EDuration="7.608478467s" totalImagesPullingTime="3.48782167s" totalInitContainerRuntime="1.087301201s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:16.606811398 +0000 UTC m=+373.106204198" watchObservedRunningTime="2026-09-14 07:55:16.608478467 +0000 UTC m=+373.107871316"
Sep 14 07:55:27.594091 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:27.594057    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-ttn8h"
Sep 14 07:55:35.677229 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:35.677198    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65bf895c7c-n8vnn"]
Sep 14 07:55:40.392385 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392345    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g6z28"]
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392608    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="extract"
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392619    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="extract"
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392636    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="pull"
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392642    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="pull"
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392648    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="util"
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392653    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="util"
Sep 14 07:55:40.392750 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.392700    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d7be046-493a-41f6-876a-ba130d3a6d5d" containerName="extract"
Sep 14 07:55:40.399471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.399448    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:40.401644 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.401620    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 14 07:55:40.401753 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.401682    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dhfcm\""
Sep 14 07:55:40.408007 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.407981    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g6z28"]
Sep 14 07:55:40.460874 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.460843    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcpmz\" (UniqueName: \"kubernetes.io/projected/9662987c-4423-4a86-b32e-5e6b726d2d3a-kube-api-access-rcpmz\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:40.460874 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.460886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9662987c-4423-4a86-b32e-5e6b726d2d3a-cert\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:40.562100 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.562067    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9662987c-4423-4a86-b32e-5e6b726d2d3a-cert\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:40.562280 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.562158    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcpmz\" (UniqueName: \"kubernetes.io/projected/9662987c-4423-4a86-b32e-5e6b726d2d3a-kube-api-access-rcpmz\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:40.562280 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:55:40.562216    2556 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 14 07:55:40.562372 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:55:40.562291    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9662987c-4423-4a86-b32e-5e6b726d2d3a-cert podName:9662987c-4423-4a86-b32e-5e6b726d2d3a nodeName:}" failed. No retries permitted until 2026-09-14 07:55:41.0622737 +0000 UTC m=+397.561666455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9662987c-4423-4a86-b32e-5e6b726d2d3a-cert") pod "odh-model-controller-7998c67dc4-g6z28" (UID: "9662987c-4423-4a86-b32e-5e6b726d2d3a") : secret "odh-model-controller-webhook-cert" not found
Sep 14 07:55:40.570210 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.570183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcpmz\" (UniqueName: \"kubernetes.io/projected/9662987c-4423-4a86-b32e-5e6b726d2d3a-kube-api-access-rcpmz\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:40.784887 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.784844    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wkflq"]
Sep 14 07:55:40.787208 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.787184    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:40.789191 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.789165    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 14 07:55:40.789320 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.789249    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jzlgx\""
Sep 14 07:55:40.796540 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.796514    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wkflq"]
Sep 14 07:55:40.865370 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.865335    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/054dc377-2db5-4561-ada1-42c434bc070e-tls-certs\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:40.865539 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.865376    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvmpg\" (UniqueName: \"kubernetes.io/projected/054dc377-2db5-4561-ada1-42c434bc070e-kube-api-access-rvmpg\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:40.966590 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.966532    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvmpg\" (UniqueName: \"kubernetes.io/projected/054dc377-2db5-4561-ada1-42c434bc070e-kube-api-access-rvmpg\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:40.966798 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.966678    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/054dc377-2db5-4561-ada1-42c434bc070e-tls-certs\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:40.966798 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:55:40.966788    2556 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 14 07:55:40.966918 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:55:40.966853    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/054dc377-2db5-4561-ada1-42c434bc070e-tls-certs podName:054dc377-2db5-4561-ada1-42c434bc070e nodeName:}" failed. No retries permitted until 2026-09-14 07:55:41.466833868 +0000 UTC m=+397.966226640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/054dc377-2db5-4561-ada1-42c434bc070e-tls-certs") pod "model-serving-api-54d89bd5fc-wkflq" (UID: "054dc377-2db5-4561-ada1-42c434bc070e") : secret "model-serving-api-tls" not found
Sep 14 07:55:40.974907 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:40.974870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvmpg\" (UniqueName: \"kubernetes.io/projected/054dc377-2db5-4561-ada1-42c434bc070e-kube-api-access-rvmpg\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:41.067622 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.067541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9662987c-4423-4a86-b32e-5e6b726d2d3a-cert\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:41.069815 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.069787    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9662987c-4423-4a86-b32e-5e6b726d2d3a-cert\") pod \"odh-model-controller-7998c67dc4-g6z28\" (UID: \"9662987c-4423-4a86-b32e-5e6b726d2d3a\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:41.312021 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.311958    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:41.470283 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.470245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/054dc377-2db5-4561-ada1-42c434bc070e-tls-certs\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:41.472594 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.472566    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/054dc377-2db5-4561-ada1-42c434bc070e-tls-certs\") pod \"model-serving-api-54d89bd5fc-wkflq\" (UID: \"054dc377-2db5-4561-ada1-42c434bc070e\") " pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:41.581454 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.581390    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g6z28"]
Sep 14 07:55:41.585371 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:55:41.585341    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9662987c_4423_4a86_b32e_5e6b726d2d3a.slice/crio-6f5c3c44a24a2313e2e9ec3f52dfe0772feb2b7e70019370a075220313d9a239 WatchSource:0}: Error finding container 6f5c3c44a24a2313e2e9ec3f52dfe0772feb2b7e70019370a075220313d9a239: Status 404 returned error can't find the container with id 6f5c3c44a24a2313e2e9ec3f52dfe0772feb2b7e70019370a075220313d9a239
Sep 14 07:55:41.696103 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.696063    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28" event={"ID":"9662987c-4423-4a86-b32e-5e6b726d2d3a","Type":"ContainerStarted","Data":"6f5c3c44a24a2313e2e9ec3f52dfe0772feb2b7e70019370a075220313d9a239"}
Sep 14 07:55:41.699282 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.699261    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:41.975047 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:41.975011    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wkflq"]
Sep 14 07:55:41.978618 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:55:41.978585    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod054dc377_2db5_4561_ada1_42c434bc070e.slice/crio-17a2c42467a6105dd8a3bb742ab2e6befad4b22836625343addbb55bbcad7457 WatchSource:0}: Error finding container 17a2c42467a6105dd8a3bb742ab2e6befad4b22836625343addbb55bbcad7457: Status 404 returned error can't find the container with id 17a2c42467a6105dd8a3bb742ab2e6befad4b22836625343addbb55bbcad7457
Sep 14 07:55:42.701431 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:42.701392    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq" event={"ID":"054dc377-2db5-4561-ada1-42c434bc070e","Type":"ContainerStarted","Data":"17a2c42467a6105dd8a3bb742ab2e6befad4b22836625343addbb55bbcad7457"}
Sep 14 07:55:45.716167 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:45.716135    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28" event={"ID":"9662987c-4423-4a86-b32e-5e6b726d2d3a","Type":"ContainerStarted","Data":"a40b5ddea6c9759afb1412fad384610147cf3a84d1d5b60e09885225a95c0ef2"}
Sep 14 07:55:45.716560 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:45.716234    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:45.717686 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:45.717654    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq" event={"ID":"054dc377-2db5-4561-ada1-42c434bc070e","Type":"ContainerStarted","Data":"4dd4ff75287e575b4b162b03990673e23ee8fbdca55fa864b33246cb53a89e09"}
Sep 14 07:55:45.717828 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:45.717702    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:55:45.730501 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:45.730461    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28" podStartSLOduration=2.057795051 podStartE2EDuration="5.730447948s" totalImagesPullingTime="3.672652897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:45.729449135 +0000 UTC m=+402.228841962" watchObservedRunningTime="2026-09-14 07:55:45.730447948 +0000 UTC m=+402.229840730"
Sep 14 07:55:45.746492 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:45.746454    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq" podStartSLOduration=2.459404229 podStartE2EDuration="5.746441513s" totalImagesPullingTime="3.287037284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:45.745273165 +0000 UTC m=+402.244665958" watchObservedRunningTime="2026-09-14 07:55:45.746441513 +0000 UTC m=+402.245834290"
Sep 14 07:55:49.926311 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:49.926273    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"]
Sep 14 07:55:49.928787 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:49.928763    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:49.931021 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:49.930996    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 14 07:55:49.931153 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:49.931100    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zngbb\""
Sep 14 07:55:49.939556 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:49.939516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"]
Sep 14 07:55:50.042652 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.042617    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-cert\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.042823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.042684    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfr4c\" (UniqueName: \"kubernetes.io/projected/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-kube-api-access-nfr4c\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.144142 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.144104    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-cert\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.144333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.144173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfr4c\" (UniqueName: \"kubernetes.io/projected/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-kube-api-access-nfr4c\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.144333 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:55:50.144265    2556 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 14 07:55:50.144441 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:55:50.144361    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-cert podName:ed06b2ab-ee0b-43d2-8308-b01b4eb31dce nodeName:}" failed. No retries permitted until 2026-09-14 07:55:50.64434143 +0000 UTC m=+407.143734196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-cert") pod "kserve-controller-manager-d6f4b65b8-td4m4" (UID: "ed06b2ab-ee0b-43d2-8308-b01b4eb31dce") : secret "kserve-webhook-server-cert" not found
Sep 14 07:55:50.157621 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.157587    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfr4c\" (UniqueName: \"kubernetes.io/projected/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-kube-api-access-nfr4c\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.647669 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.647635    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-cert\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.649974 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.649944    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed06b2ab-ee0b-43d2-8308-b01b4eb31dce-cert\") pod \"kserve-controller-manager-d6f4b65b8-td4m4\" (UID: \"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:50.843073 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:50.843023    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:51.114854 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:51.114829    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"]
Sep 14 07:55:51.117336 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:55:51.117308    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded06b2ab_ee0b_43d2_8308_b01b4eb31dce.slice/crio-714bea39b93454b3022795addf0cd29cdcbe5e0bc521022bc014751d2c128d73 WatchSource:0}: Error finding container 714bea39b93454b3022795addf0cd29cdcbe5e0bc521022bc014751d2c128d73: Status 404 returned error can't find the container with id 714bea39b93454b3022795addf0cd29cdcbe5e0bc521022bc014751d2c128d73
Sep 14 07:55:51.746068 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:51.746028    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4" event={"ID":"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce","Type":"ContainerStarted","Data":"714bea39b93454b3022795addf0cd29cdcbe5e0bc521022bc014751d2c128d73"}
Sep 14 07:55:53.762461 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:53.762360    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4" event={"ID":"ed06b2ab-ee0b-43d2-8308-b01b4eb31dce","Type":"ContainerStarted","Data":"54a0875895ce6b7f6df083628d48bc6a235f5e3ef9f8c3d5d0e48ef0bd1c05c3"}
Sep 14 07:55:53.762840 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:53.762515    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:55:53.781827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:53.781778    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4" podStartSLOduration=2.38913066 podStartE2EDuration="4.781762918s" totalImagesPullingTime="2.392632258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:53.779936878 +0000 UTC m=+410.279329679" watchObservedRunningTime="2026-09-14 07:55:53.781762918 +0000 UTC m=+410.281155696"
Sep 14 07:55:56.724399 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:56.724369    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-g6z28"
Sep 14 07:55:56.726607 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:55:56.726585    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-wkflq"
Sep 14 07:56:00.700433 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:00.700365    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65bf895c7c-n8vnn" podUID="f519d385-49d3-42d3-9340-9c0efe3142e0" containerName="console" containerID="cri-o://515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901" gracePeriod=15
Sep 14 07:56:00.935918 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:00.935894    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:56:01.034101 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/f519d385-49d3-42d3-9340-9c0efe3142e0-kube-api-access-4lt5n\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034101 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034052    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-oauth-config\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034101 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034094    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-service-ca\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034365 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034131    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-oauth-serving-cert\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034365 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034155    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-serving-cert\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034365 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034179    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-console-config\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034365 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034227    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-trusted-ca-bundle\") pod \"f519d385-49d3-42d3-9340-9c0efe3142e0\" (UID: \"f519d385-49d3-42d3-9340-9c0efe3142e0\") "
Sep 14 07:56:01.034614 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034576    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:56:01.034689 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034614    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-service-ca" (OuterVolumeSpecName: "service-ca") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:56:01.034745 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034697    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-console-config" (OuterVolumeSpecName: "console-config") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:56:01.034745 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.034710    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:56:01.036349 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.036320    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f519d385-49d3-42d3-9340-9c0efe3142e0-kube-api-access-4lt5n" (OuterVolumeSpecName: "kube-api-access-4lt5n") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "kube-api-access-4lt5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:01.036462 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.036412    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:56:01.036462 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.036426    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f519d385-49d3-42d3-9340-9c0efe3142e0" (UID: "f519d385-49d3-42d3-9340-9c0efe3142e0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:56:01.134672 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134637    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/f519d385-49d3-42d3-9340-9c0efe3142e0-kube-api-access-4lt5n\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.134672 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134663    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.134672 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134673    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.134672 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134681    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.134909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134692    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f519d385-49d3-42d3-9340-9c0efe3142e0-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.134909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134701    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.134909 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.134708    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f519d385-49d3-42d3-9340-9c0efe3142e0-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:01.800176 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.800141    2556 generic.go:356] "Generic (PLEG): container finished" podID="f519d385-49d3-42d3-9340-9c0efe3142e0" containerID="515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901" exitCode=0
Sep 14 07:56:01.800578 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.800182    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65bf895c7c-n8vnn"
Sep 14 07:56:01.800578 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.800225    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65bf895c7c-n8vnn" event={"ID":"f519d385-49d3-42d3-9340-9c0efe3142e0","Type":"ContainerDied","Data":"515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901"}
Sep 14 07:56:01.800578 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.800272    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65bf895c7c-n8vnn" event={"ID":"f519d385-49d3-42d3-9340-9c0efe3142e0","Type":"ContainerDied","Data":"c1e11791952987d92b1cffa73d747ed42c5821592129b4f3abaee1a6d6a884a5"}
Sep 14 07:56:01.800578 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.800298    2556 scope.go:117] "RemoveContainer" containerID="515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901"
Sep 14 07:56:01.809374 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.809358    2556 scope.go:117] "RemoveContainer" containerID="515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901"
Sep 14 07:56:01.809638 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:56:01.809616    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901\": container with ID starting with 515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901 not found: ID does not exist" containerID="515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901"
Sep 14 07:56:01.809688 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.809645    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901"} err="failed to get container status \"515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901\": rpc error: code = NotFound desc = could not find container \"515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901\": container with ID starting with 515f5479499afa1559c894a72c55ca31be656a510fa43edc6680a3d72b06a901 not found: ID does not exist"
Sep 14 07:56:01.819505 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.819478    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65bf895c7c-n8vnn"]
Sep 14 07:56:01.821118 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:01.821096    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65bf895c7c-n8vnn"]
Sep 14 07:56:02.018067 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:02.018033    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f519d385-49d3-42d3-9340-9c0efe3142e0" path="/var/lib/kubelet/pods/f519d385-49d3-42d3-9340-9c0efe3142e0/volumes"
Sep 14 07:56:24.771712 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:24.771675    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-td4m4"
Sep 14 07:56:45.423561 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.423524    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"]
Sep 14 07:56:45.423918 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.423829    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f519d385-49d3-42d3-9340-9c0efe3142e0" containerName="console"
Sep 14 07:56:45.423918 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.423840    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f519d385-49d3-42d3-9340-9c0efe3142e0" containerName="console"
Sep 14 07:56:45.423918 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.423887    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f519d385-49d3-42d3-9340-9c0efe3142e0" containerName="console"
Sep 14 07:56:45.425806 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.425789    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.428164 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.428139    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:56:45.428164 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.428154    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:56:45.428865 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.428849    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9nkq4\""
Sep 14 07:56:45.435251 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.435229    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"]
Sep 14 07:56:45.532934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.532897    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"]
Sep 14 07:56:45.535238 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.535220    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.545720 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.545693    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"]
Sep 14 07:56:45.587385 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.587356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.587546 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.587397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.587546 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.587460    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf5zl\" (UniqueName: \"kubernetes.io/projected/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-kube-api-access-mf5zl\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.621809 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.621782    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"]
Sep 14 07:56:45.624049 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.624034    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.632890 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.632866    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"]
Sep 14 07:56:45.687937 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.687863    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.687937 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.687908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5zl\" (UniqueName: \"kubernetes.io/projected/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-kube-api-access-mf5zl\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.687937 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.687937    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7xzg\" (UniqueName: \"kubernetes.io/projected/87d80e85-ba3c-4266-a3d4-4baee5782fcc-kube-api-access-b7xzg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.688180 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.687997    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.688180 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.688019    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.688180 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.688036    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.688336 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.688316    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.688373 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.688348    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.695534 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.695508    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf5zl\" (UniqueName: \"kubernetes.io/projected/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-kube-api-access-mf5zl\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.720106 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.720078    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"]
Sep 14 07:56:45.722484 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.722469    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.734520 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.734496    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"]
Sep 14 07:56:45.734621 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.734537    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:45.788458 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788429    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.788562 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788480    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd62t\" (UniqueName: \"kubernetes.io/projected/ae427c96-a087-4d95-910f-b5aa051b8ec0-kube-api-access-qd62t\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.788562 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788511    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.788648 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7xzg\" (UniqueName: \"kubernetes.io/projected/87d80e85-ba3c-4266-a3d4-4baee5782fcc-kube-api-access-b7xzg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.788710 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788693    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.788785 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788760    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.788845 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.788790    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.789088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.789059    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.799010 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.798950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7xzg\" (UniqueName: \"kubernetes.io/projected/87d80e85-ba3c-4266-a3d4-4baee5782fcc-kube-api-access-b7xzg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.844381 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.844350    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:45.889371 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfknq\" (UniqueName: \"kubernetes.io/projected/96aca3db-267c-4090-b9f5-992bc787a5e4-kube-api-access-nfknq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889382    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889463    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd62t\" (UniqueName: \"kubernetes.io/projected/ae427c96-a087-4d95-910f-b5aa051b8ec0-kube-api-access-qd62t\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889500    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889520    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889567    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889785    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.890088 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.889943    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.899554 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.899528    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd62t\" (UniqueName: \"kubernetes.io/projected/ae427c96-a087-4d95-910f-b5aa051b8ec0-kube-api-access-qd62t\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.932947 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.932920    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:45.990683 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.990648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfknq\" (UniqueName: \"kubernetes.io/projected/96aca3db-267c-4090-b9f5-992bc787a5e4-kube-api-access-nfknq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.990827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.990690    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.990827 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.990732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.991179 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.991153    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.991273 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.991203    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:45.998679 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:45.998658    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfknq\" (UniqueName: \"kubernetes.io/projected/96aca3db-267c-4090-b9f5-992bc787a5e4-kube-api-access-nfknq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:46.005931 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:46.005907    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"]
Sep 14 07:56:46.007994 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:56:46.007971    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483eaa9c_a156_4c79_93ea_58f5d2dd6c02.slice/crio-2417fdef126a48a8b208703bd824fcf001937b41143abb5b0860f7b94550a353 WatchSource:0}: Error finding container 2417fdef126a48a8b208703bd824fcf001937b41143abb5b0860f7b94550a353: Status 404 returned error can't find the container with id 2417fdef126a48a8b208703bd824fcf001937b41143abb5b0860f7b94550a353
Sep 14 07:56:46.031754 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:46.031732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:46.112356 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:46.112329    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"]
Sep 14 07:56:46.115511 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:56:46.115480    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d80e85_ba3c_4266_a3d4_4baee5782fcc.slice/crio-9eb1fd9b6d3ffad16a77a0053ac734f9e6985b6d592a5016d195aac9bdddf2c3 WatchSource:0}: Error finding container 9eb1fd9b6d3ffad16a77a0053ac734f9e6985b6d592a5016d195aac9bdddf2c3: Status 404 returned error can't find the container with id 9eb1fd9b6d3ffad16a77a0053ac734f9e6985b6d592a5016d195aac9bdddf2c3
Sep 14 07:56:46.201448 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:46.201416    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"]
Sep 14 07:56:46.203296 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:56:46.203274    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae427c96_a087_4d95_910f_b5aa051b8ec0.slice/crio-5ce3f25b6bb45577858cc2a88205f3c32c6364532713d22d871a3b4df11afd20 WatchSource:0}: Error finding container 5ce3f25b6bb45577858cc2a88205f3c32c6364532713d22d871a3b4df11afd20: Status 404 returned error can't find the container with id 5ce3f25b6bb45577858cc2a88205f3c32c6364532713d22d871a3b4df11afd20
Sep 14 07:56:46.306494 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:46.306471    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"]
Sep 14 07:56:46.334855 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:56:46.334827    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96aca3db_267c_4090_b9f5_992bc787a5e4.slice/crio-6a13469723ad431acaaf6a181cb07fc357c0f184f27d99bbb27218f7a8e2e282 WatchSource:0}: Error finding container 6a13469723ad431acaaf6a181cb07fc357c0f184f27d99bbb27218f7a8e2e282: Status 404 returned error can't find the container with id 6a13469723ad431acaaf6a181cb07fc357c0f184f27d99bbb27218f7a8e2e282
Sep 14 07:56:47.000586 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.000542    2556 generic.go:356] "Generic (PLEG): container finished" podID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerID="11f72775c4270d5f8f19b54e87c92a175071544497ed825e083c4a7d09a07ce8" exitCode=0
Sep 14 07:56:47.001065 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.000622    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl" event={"ID":"483eaa9c-a156-4c79-93ea-58f5d2dd6c02","Type":"ContainerDied","Data":"11f72775c4270d5f8f19b54e87c92a175071544497ed825e083c4a7d09a07ce8"}
Sep 14 07:56:47.001065 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.000674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl" event={"ID":"483eaa9c-a156-4c79-93ea-58f5d2dd6c02","Type":"ContainerStarted","Data":"2417fdef126a48a8b208703bd824fcf001937b41143abb5b0860f7b94550a353"}
Sep 14 07:56:47.002251 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.002228    2556 generic.go:356] "Generic (PLEG): container finished" podID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerID="bcbb7cdc9a9c5384b22412d38adc76ff692e8d40e9d8de1fc443e52a0847c87a" exitCode=0
Sep 14 07:56:47.002333 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.002311    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7" event={"ID":"96aca3db-267c-4090-b9f5-992bc787a5e4","Type":"ContainerDied","Data":"bcbb7cdc9a9c5384b22412d38adc76ff692e8d40e9d8de1fc443e52a0847c87a"}
Sep 14 07:56:47.002393 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.002360    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7" event={"ID":"96aca3db-267c-4090-b9f5-992bc787a5e4","Type":"ContainerStarted","Data":"6a13469723ad431acaaf6a181cb07fc357c0f184f27d99bbb27218f7a8e2e282"}
Sep 14 07:56:47.004005 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.003982    2556 generic.go:356] "Generic (PLEG): container finished" podID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerID="28e213ad520cc5bdc33155efa2468b2fa6421703b17a50e666968f1a069fe9ab" exitCode=0
Sep 14 07:56:47.004093 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.004063    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2" event={"ID":"87d80e85-ba3c-4266-a3d4-4baee5782fcc","Type":"ContainerDied","Data":"28e213ad520cc5bdc33155efa2468b2fa6421703b17a50e666968f1a069fe9ab"}
Sep 14 07:56:47.004153 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.004107    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2" event={"ID":"87d80e85-ba3c-4266-a3d4-4baee5782fcc","Type":"ContainerStarted","Data":"9eb1fd9b6d3ffad16a77a0053ac734f9e6985b6d592a5016d195aac9bdddf2c3"}
Sep 14 07:56:47.005752 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.005731    2556 generic.go:356] "Generic (PLEG): container finished" podID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerID="66b2c64e80d015e86d0a18b9cac7ee75f9dd72d6befa1fc2e590013f64088e7d" exitCode=0
Sep 14 07:56:47.005859 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.005799    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc" event={"ID":"ae427c96-a087-4d95-910f-b5aa051b8ec0","Type":"ContainerDied","Data":"66b2c64e80d015e86d0a18b9cac7ee75f9dd72d6befa1fc2e590013f64088e7d"}
Sep 14 07:56:47.005859 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:47.005835    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc" event={"ID":"ae427c96-a087-4d95-910f-b5aa051b8ec0","Type":"ContainerStarted","Data":"5ce3f25b6bb45577858cc2a88205f3c32c6364532713d22d871a3b4df11afd20"}
Sep 14 07:56:48.012288 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:48.012249    2556 generic.go:356] "Generic (PLEG): container finished" podID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerID="388577a703d5ca23652b583ab268d0d44e07a22b1f42dba3b143b4efbfaea629" exitCode=0
Sep 14 07:56:48.014435 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:48.014409    2556 generic.go:356] "Generic (PLEG): container finished" podID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerID="59ec093cb471fa44eeb800c6536f3da09aad1dd595a3873dcb2f2c7f00367a21" exitCode=0
Sep 14 07:56:48.016628 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:48.016605    2556 generic.go:356] "Generic (PLEG): container finished" podID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerID="3b5a86a82e03356438b874aabb5508e53f934165ca54ce6f6d8fc3b0c5944183" exitCode=0
Sep 14 07:56:48.019409 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:48.019381    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl" event={"ID":"483eaa9c-a156-4c79-93ea-58f5d2dd6c02","Type":"ContainerDied","Data":"388577a703d5ca23652b583ab268d0d44e07a22b1f42dba3b143b4efbfaea629"}
Sep 14 07:56:48.019459 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:48.019439    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7" event={"ID":"96aca3db-267c-4090-b9f5-992bc787a5e4","Type":"ContainerDied","Data":"59ec093cb471fa44eeb800c6536f3da09aad1dd595a3873dcb2f2c7f00367a21"}
Sep 14 07:56:48.019498 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:48.019465    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2" event={"ID":"87d80e85-ba3c-4266-a3d4-4baee5782fcc","Type":"ContainerDied","Data":"3b5a86a82e03356438b874aabb5508e53f934165ca54ce6f6d8fc3b0c5944183"}
Sep 14 07:56:49.025349 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:49.025313    2556 generic.go:356] "Generic (PLEG): container finished" podID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerID="f6a1defec421c8e893cca42bbe2c53e4d073cc596be9d2465c0bbc6c77450e9c" exitCode=0
Sep 14 07:56:49.025776 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:49.025374    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl" event={"ID":"483eaa9c-a156-4c79-93ea-58f5d2dd6c02","Type":"ContainerDied","Data":"f6a1defec421c8e893cca42bbe2c53e4d073cc596be9d2465c0bbc6c77450e9c"}
Sep 14 07:56:49.027447 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:49.027423    2556 generic.go:356] "Generic (PLEG): container finished" podID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerID="c3d75cf7f0d393aa795080c45a7e34e5d5c9ed039a11f2d3fec7e11d44353cfd" exitCode=0
Sep 14 07:56:49.027569 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:49.027498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7" event={"ID":"96aca3db-267c-4090-b9f5-992bc787a5e4","Type":"ContainerDied","Data":"c3d75cf7f0d393aa795080c45a7e34e5d5c9ed039a11f2d3fec7e11d44353cfd"}
Sep 14 07:56:49.029643 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:49.029622    2556 generic.go:356] "Generic (PLEG): container finished" podID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerID="2ff7c1e002a03d7067d9c41f1a9669f2fa2cb095901d26686aea0571382440e4" exitCode=0
Sep 14 07:56:49.029722 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:49.029705    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2" event={"ID":"87d80e85-ba3c-4266-a3d4-4baee5782fcc","Type":"ContainerDied","Data":"2ff7c1e002a03d7067d9c41f1a9669f2fa2cb095901d26686aea0571382440e4"}
Sep 14 07:56:50.169203 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.169173    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:50.210829 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.210808    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:50.213726 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.213702    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:50.224490 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224459    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7xzg\" (UniqueName: \"kubernetes.io/projected/87d80e85-ba3c-4266-a3d4-4baee5782fcc-kube-api-access-b7xzg\") pod \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") "
Sep 14 07:56:50.224615 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224538    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-util\") pod \"96aca3db-267c-4090-b9f5-992bc787a5e4\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") "
Sep 14 07:56:50.224615 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224566    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-util\") pod \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") "
Sep 14 07:56:50.224615 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224594    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-bundle\") pod \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") "
Sep 14 07:56:50.224823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224614    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-util\") pod \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\" (UID: \"87d80e85-ba3c-4266-a3d4-4baee5782fcc\") "
Sep 14 07:56:50.224823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224657    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf5zl\" (UniqueName: \"kubernetes.io/projected/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-kube-api-access-mf5zl\") pod \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") "
Sep 14 07:56:50.224823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224693    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfknq\" (UniqueName: \"kubernetes.io/projected/96aca3db-267c-4090-b9f5-992bc787a5e4-kube-api-access-nfknq\") pod \"96aca3db-267c-4090-b9f5-992bc787a5e4\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") "
Sep 14 07:56:50.224823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224741    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-bundle\") pod \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\" (UID: \"483eaa9c-a156-4c79-93ea-58f5d2dd6c02\") "
Sep 14 07:56:50.224823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.224774    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-bundle\") pod \"96aca3db-267c-4090-b9f5-992bc787a5e4\" (UID: \"96aca3db-267c-4090-b9f5-992bc787a5e4\") "
Sep 14 07:56:50.225226 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.225098    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-bundle" (OuterVolumeSpecName: "bundle") pod "87d80e85-ba3c-4266-a3d4-4baee5782fcc" (UID: "87d80e85-ba3c-4266-a3d4-4baee5782fcc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:50.226003 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.225928    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-bundle" (OuterVolumeSpecName: "bundle") pod "483eaa9c-a156-4c79-93ea-58f5d2dd6c02" (UID: "483eaa9c-a156-4c79-93ea-58f5d2dd6c02"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:50.226003 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.225956    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-bundle" (OuterVolumeSpecName: "bundle") pod "96aca3db-267c-4090-b9f5-992bc787a5e4" (UID: "96aca3db-267c-4090-b9f5-992bc787a5e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:50.227747 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.227618    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87d80e85-ba3c-4266-a3d4-4baee5782fcc-kube-api-access-b7xzg" (OuterVolumeSpecName: "kube-api-access-b7xzg") pod "87d80e85-ba3c-4266-a3d4-4baee5782fcc" (UID: "87d80e85-ba3c-4266-a3d4-4baee5782fcc"). InnerVolumeSpecName "kube-api-access-b7xzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:50.232830 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.228384    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-kube-api-access-mf5zl" (OuterVolumeSpecName: "kube-api-access-mf5zl") pod "483eaa9c-a156-4c79-93ea-58f5d2dd6c02" (UID: "483eaa9c-a156-4c79-93ea-58f5d2dd6c02"). InnerVolumeSpecName "kube-api-access-mf5zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:50.232830 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.232757    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-util" (OuterVolumeSpecName: "util") pod "96aca3db-267c-4090-b9f5-992bc787a5e4" (UID: "96aca3db-267c-4090-b9f5-992bc787a5e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:50.234386 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.233681    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-util" (OuterVolumeSpecName: "util") pod "87d80e85-ba3c-4266-a3d4-4baee5782fcc" (UID: "87d80e85-ba3c-4266-a3d4-4baee5782fcc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:50.236822 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.236785    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96aca3db-267c-4090-b9f5-992bc787a5e4-kube-api-access-nfknq" (OuterVolumeSpecName: "kube-api-access-nfknq") pod "96aca3db-267c-4090-b9f5-992bc787a5e4" (UID: "96aca3db-267c-4090-b9f5-992bc787a5e4"). InnerVolumeSpecName "kube-api-access-nfknq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:50.240458 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.240436    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-util" (OuterVolumeSpecName: "util") pod "483eaa9c-a156-4c79-93ea-58f5d2dd6c02" (UID: "483eaa9c-a156-4c79-93ea-58f5d2dd6c02"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325729    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325758    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7xzg\" (UniqueName: \"kubernetes.io/projected/87d80e85-ba3c-4266-a3d4-4baee5782fcc-kube-api-access-b7xzg\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325770    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96aca3db-267c-4090-b9f5-992bc787a5e4-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325779    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325787    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325794    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87d80e85-ba3c-4266-a3d4-4baee5782fcc-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325801    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf5zl\" (UniqueName: \"kubernetes.io/projected/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-kube-api-access-mf5zl\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325809    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfknq\" (UniqueName: \"kubernetes.io/projected/96aca3db-267c-4090-b9f5-992bc787a5e4-kube-api-access-nfknq\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:50.325812 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:50.325818    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/483eaa9c-a156-4c79-93ea-58f5d2dd6c02-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:51.039713 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.039681    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7"
Sep 14 07:56:51.039880 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.039681    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ankqh7" event={"ID":"96aca3db-267c-4090-b9f5-992bc787a5e4","Type":"ContainerDied","Data":"6a13469723ad431acaaf6a181cb07fc357c0f184f27d99bbb27218f7a8e2e282"}
Sep 14 07:56:51.039880 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.039820    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a13469723ad431acaaf6a181cb07fc357c0f184f27d99bbb27218f7a8e2e282"
Sep 14 07:56:51.041571 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.041548    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2"
Sep 14 07:56:51.041695 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.041585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ztrc2" event={"ID":"87d80e85-ba3c-4266-a3d4-4baee5782fcc","Type":"ContainerDied","Data":"9eb1fd9b6d3ffad16a77a0053ac734f9e6985b6d592a5016d195aac9bdddf2c3"}
Sep 14 07:56:51.041695 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.041623    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9eb1fd9b6d3ffad16a77a0053ac734f9e6985b6d592a5016d195aac9bdddf2c3"
Sep 14 07:56:51.043637 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.043616    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl" event={"ID":"483eaa9c-a156-4c79-93ea-58f5d2dd6c02","Type":"ContainerDied","Data":"2417fdef126a48a8b208703bd824fcf001937b41143abb5b0860f7b94550a353"}
Sep 14 07:56:51.043717 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.043651    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2417fdef126a48a8b208703bd824fcf001937b41143abb5b0860f7b94550a353"
Sep 14 07:56:51.043717 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:51.043668    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mdfcl"
Sep 14 07:56:55.064277 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:55.064240    2556 generic.go:356] "Generic (PLEG): container finished" podID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerID="459db1390ec0ec5f4291f47ce58f5e269151d7ec87cdcf8cc01e467b736b36ee" exitCode=0
Sep 14 07:56:55.064717 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:55.064317    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc" event={"ID":"ae427c96-a087-4d95-910f-b5aa051b8ec0","Type":"ContainerDied","Data":"459db1390ec0ec5f4291f47ce58f5e269151d7ec87cdcf8cc01e467b736b36ee"}
Sep 14 07:56:56.071051 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:56.071019    2556 generic.go:356] "Generic (PLEG): container finished" podID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerID="f160b175f37e9c52a2c42c73ac26dc4f778a768912d491707eadedd3f1c4fc33" exitCode=0
Sep 14 07:56:56.071571 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:56.071061    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc" event={"ID":"ae427c96-a087-4d95-910f-b5aa051b8ec0","Type":"ContainerDied","Data":"f160b175f37e9c52a2c42c73ac26dc4f778a768912d491707eadedd3f1c4fc33"}
Sep 14 07:56:57.200406 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.200379    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:57.282716 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.282678    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-bundle\") pod \"ae427c96-a087-4d95-910f-b5aa051b8ec0\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") "
Sep 14 07:56:57.282890 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.282774    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-util\") pod \"ae427c96-a087-4d95-910f-b5aa051b8ec0\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") "
Sep 14 07:56:57.282890 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.282799    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd62t\" (UniqueName: \"kubernetes.io/projected/ae427c96-a087-4d95-910f-b5aa051b8ec0-kube-api-access-qd62t\") pod \"ae427c96-a087-4d95-910f-b5aa051b8ec0\" (UID: \"ae427c96-a087-4d95-910f-b5aa051b8ec0\") "
Sep 14 07:56:57.283285 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.283259    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-bundle" (OuterVolumeSpecName: "bundle") pod "ae427c96-a087-4d95-910f-b5aa051b8ec0" (UID: "ae427c96-a087-4d95-910f-b5aa051b8ec0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:57.284838 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.284816    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae427c96-a087-4d95-910f-b5aa051b8ec0-kube-api-access-qd62t" (OuterVolumeSpecName: "kube-api-access-qd62t") pod "ae427c96-a087-4d95-910f-b5aa051b8ec0" (UID: "ae427c96-a087-4d95-910f-b5aa051b8ec0"). InnerVolumeSpecName "kube-api-access-qd62t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:57.287236 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.287214    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-util" (OuterVolumeSpecName: "util") pod "ae427c96-a087-4d95-910f-b5aa051b8ec0" (UID: "ae427c96-a087-4d95-910f-b5aa051b8ec0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:56:57.383302 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.383224    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:57.383302 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.383255    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae427c96-a087-4d95-910f-b5aa051b8ec0-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:57.383302 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:57.383264    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd62t\" (UniqueName: \"kubernetes.io/projected/ae427c96-a087-4d95-910f-b5aa051b8ec0-kube-api-access-qd62t\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:56:58.081781 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:58.081751    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc"
Sep 14 07:56:58.081934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:58.081750    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cckczc" event={"ID":"ae427c96-a087-4d95-910f-b5aa051b8ec0","Type":"ContainerDied","Data":"5ce3f25b6bb45577858cc2a88205f3c32c6364532713d22d871a3b4df11afd20"}
Sep 14 07:56:58.081934 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:56:58.081871    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ce3f25b6bb45577858cc2a88205f3c32c6364532713d22d871a3b4df11afd20"
Sep 14 07:57:03.310636 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.310596    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"]
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311064    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="pull"
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311083    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="pull"
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311101    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="util"
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311109    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="util"
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311123    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="util"
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311130    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="util"
Sep 14 07:57:03.311146 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311149    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311158    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311168    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311176    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311191    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="util"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311199    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="util"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311208    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="pull"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311216    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="pull"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311225    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="pull"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311234    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="pull"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311246    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="util"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311254    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="util"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311265    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="pull"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311272    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="pull"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311282    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311291    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311305    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311313    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311402    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="96aca3db-267c-4090-b9f5-992bc787a5e4" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311414    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="87d80e85-ba3c-4266-a3d4-4baee5782fcc" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311422    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="483eaa9c-a156-4c79-93ea-58f5d2dd6c02" containerName="extract"
Sep 14 07:57:03.311485 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.311432    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae427c96-a087-4d95-910f-b5aa051b8ec0" containerName="extract"
Sep 14 07:57:03.314769 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.314749    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:03.317125 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.317104    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 07:57:03.317195 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.317151    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 07:57:03.317260 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.317212    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rdvhf\""
Sep 14 07:57:03.324071 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.324048    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"]
Sep 14 07:57:03.427500 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.427475    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dmh8\" (UniqueName: \"kubernetes.io/projected/6ffdaeae-caac-428e-bdfe-f2fd8618166a-kube-api-access-2dmh8\") pod \"limitador-operator-controller-manager-74495c69f8-sh8zf\" (UID: \"6ffdaeae-caac-428e-bdfe-f2fd8618166a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:03.528466 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.528426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dmh8\" (UniqueName: \"kubernetes.io/projected/6ffdaeae-caac-428e-bdfe-f2fd8618166a-kube-api-access-2dmh8\") pod \"limitador-operator-controller-manager-74495c69f8-sh8zf\" (UID: \"6ffdaeae-caac-428e-bdfe-f2fd8618166a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:03.539014 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.538981    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dmh8\" (UniqueName: \"kubernetes.io/projected/6ffdaeae-caac-428e-bdfe-f2fd8618166a-kube-api-access-2dmh8\") pod \"limitador-operator-controller-manager-74495c69f8-sh8zf\" (UID: \"6ffdaeae-caac-428e-bdfe-f2fd8618166a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:03.624697 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.624620    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:03.901482 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:03.901459    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"]
Sep 14 07:57:03.903641 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:57:03.903613    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffdaeae_caac_428e_bdfe_f2fd8618166a.slice/crio-351a1f00d2606543a8e6a5cffa6901c3af5cfd354f7560998e6a6ed08d4248fe WatchSource:0}: Error finding container 351a1f00d2606543a8e6a5cffa6901c3af5cfd354f7560998e6a6ed08d4248fe: Status 404 returned error can't find the container with id 351a1f00d2606543a8e6a5cffa6901c3af5cfd354f7560998e6a6ed08d4248fe
Sep 14 07:57:04.113825 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:04.113787    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf" event={"ID":"6ffdaeae-caac-428e-bdfe-f2fd8618166a","Type":"ContainerStarted","Data":"351a1f00d2606543a8e6a5cffa6901c3af5cfd354f7560998e6a6ed08d4248fe"}
Sep 14 07:57:07.133032 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:07.132993    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf" event={"ID":"6ffdaeae-caac-428e-bdfe-f2fd8618166a","Type":"ContainerStarted","Data":"992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb"}
Sep 14 07:57:07.133632 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:07.133059    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:07.150555 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:07.150508    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf" podStartSLOduration=1.469887392 podStartE2EDuration="4.15049218s" totalImagesPullingTime="2.680604788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:07.148503112 +0000 UTC m=+483.647895917" watchObservedRunningTime="2026-09-14 07:57:07.15049218 +0000 UTC m=+483.649884957"
Sep 14 07:57:18.139714 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:18.139681    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:19.603686 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.603647    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"]
Sep 14 07:57:19.607255 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.607233    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.610062 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.610035    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 14 07:57:19.610183 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.610070    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 14 07:57:19.610409 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.610392    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rfj9r\""
Sep 14 07:57:19.616419 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.616396    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"]
Sep 14 07:57:19.664717 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.664670    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bksr\" (UniqueName: \"kubernetes.io/projected/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-kube-api-access-6bksr\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.664884 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.664745    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.664884 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.664840    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.766008 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.765939    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bksr\" (UniqueName: \"kubernetes.io/projected/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-kube-api-access-6bksr\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.766198 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.766043    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.766198 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.766118    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.766311 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:19.766226    2556 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 14 07:57:19.766361 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:19.766319    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-plugin-serving-cert podName:c7c37f8f-c4bf-46f2-8404-d5421d8594d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:57:20.266296141 +0000 UTC m=+496.765688912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-jtxtz" (UID: "c7c37f8f-c4bf-46f2-8404-d5421d8594d7") : secret "plugin-serving-cert" not found
Sep 14 07:57:19.766757 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.766729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:19.775069 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:19.775046    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bksr\" (UniqueName: \"kubernetes.io/projected/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-kube-api-access-6bksr\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:20.269728 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:20.269696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:20.272248 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:20.272226    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c37f8f-c4bf-46f2-8404-d5421d8594d7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jtxtz\" (UID: \"c7c37f8f-c4bf-46f2-8404-d5421d8594d7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:20.517145 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:20.517112    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"
Sep 14 07:57:20.789661 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:20.789590    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz"]
Sep 14 07:57:20.792546 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:57:20.792515    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7c37f8f_c4bf_46f2_8404_d5421d8594d7.slice/crio-6fbb2279ce2d7aff2738e3ecad91348da541807cfb29f86166795b72c334f9ba WatchSource:0}: Error finding container 6fbb2279ce2d7aff2738e3ecad91348da541807cfb29f86166795b72c334f9ba: Status 404 returned error can't find the container with id 6fbb2279ce2d7aff2738e3ecad91348da541807cfb29f86166795b72c334f9ba
Sep 14 07:57:21.204582 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:21.204545    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz" event={"ID":"c7c37f8f-c4bf-46f2-8404-d5421d8594d7","Type":"ContainerStarted","Data":"6fbb2279ce2d7aff2738e3ecad91348da541807cfb29f86166795b72c334f9ba"}
Sep 14 07:57:27.238872 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:27.238832    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz" event={"ID":"c7c37f8f-c4bf-46f2-8404-d5421d8594d7","Type":"ContainerStarted","Data":"b70dec769297755c862279501b970fdbb0f48e85745d486828b463ee09e7e69e"}
Sep 14 07:57:27.259885 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:27.259833    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jtxtz" podStartSLOduration=2.137826155 podStartE2EDuration="8.259817s" totalImagesPullingTime="6.121990845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:27.257916163 +0000 UTC m=+503.757308943" watchObservedRunningTime="2026-09-14 07:57:27.259817 +0000 UTC m=+503.759209777"
Sep 14 07:57:30.511357 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.511321    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"]
Sep 14 07:57:30.511816 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.511569    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" containerName="manager" containerID="cri-o://992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb" gracePeriod=2
Sep 14 07:57:30.520546 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.520517    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"]
Sep 14 07:57:30.543389 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.543361    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"]
Sep 14 07:57:30.543676 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.543663    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" containerName="manager"
Sep 14 07:57:30.543734 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.543678    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" containerName="manager"
Sep 14 07:57:30.543768 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.543741    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" containerName="manager"
Sep 14 07:57:30.546613 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.546593    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:57:30.548819 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:30.548790    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sh8zf\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:30.559016 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.558991    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"]
Sep 14 07:57:30.652347 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.652312    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fvzk\" (UniqueName: \"kubernetes.io/projected/e6e14278-6f9d-43f2-b812-59d052d46258-kube-api-access-7fvzk\") pod \"limitador-operator-controller-manager-74495c69f8-xkkbf\" (UID: \"e6e14278-6f9d-43f2-b812-59d052d46258\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:57:30.740487 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.740461    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:30.742552 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:30.742526    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sh8zf\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:30.753153 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.753127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fvzk\" (UniqueName: \"kubernetes.io/projected/e6e14278-6f9d-43f2-b812-59d052d46258-kube-api-access-7fvzk\") pod \"limitador-operator-controller-manager-74495c69f8-xkkbf\" (UID: \"e6e14278-6f9d-43f2-b812-59d052d46258\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:57:30.761775 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.761723    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fvzk\" (UniqueName: \"kubernetes.io/projected/e6e14278-6f9d-43f2-b812-59d052d46258-kube-api-access-7fvzk\") pod \"limitador-operator-controller-manager-74495c69f8-xkkbf\" (UID: \"e6e14278-6f9d-43f2-b812-59d052d46258\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:57:30.854209 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.854174    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dmh8\" (UniqueName: \"kubernetes.io/projected/6ffdaeae-caac-428e-bdfe-f2fd8618166a-kube-api-access-2dmh8\") pod \"6ffdaeae-caac-428e-bdfe-f2fd8618166a\" (UID: \"6ffdaeae-caac-428e-bdfe-f2fd8618166a\") "
Sep 14 07:57:30.856257 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.856235    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ffdaeae-caac-428e-bdfe-f2fd8618166a-kube-api-access-2dmh8" (OuterVolumeSpecName: "kube-api-access-2dmh8") pod "6ffdaeae-caac-428e-bdfe-f2fd8618166a" (UID: "6ffdaeae-caac-428e-bdfe-f2fd8618166a"). InnerVolumeSpecName "kube-api-access-2dmh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:57:30.893718 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.893675    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:57:30.955609 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:30.955575    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dmh8\" (UniqueName: \"kubernetes.io/projected/6ffdaeae-caac-428e-bdfe-f2fd8618166a-kube-api-access-2dmh8\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:57:31.165755 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.165730    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"]
Sep 14 07:57:31.168665 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:57:31.168641    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e14278_6f9d_43f2_b812_59d052d46258.slice/crio-16d1818a51445fce345d68a028abcdbdb239c0dba4f0b1517ff63148a5f13b10 WatchSource:0}: Error finding container 16d1818a51445fce345d68a028abcdbdb239c0dba4f0b1517ff63148a5f13b10: Status 404 returned error can't find the container with id 16d1818a51445fce345d68a028abcdbdb239c0dba4f0b1517ff63148a5f13b10
Sep 14 07:57:31.261788 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.261749    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf" event={"ID":"e6e14278-6f9d-43f2-b812-59d052d46258","Type":"ContainerStarted","Data":"d4dadf85921243699beec4ebcf9c0fbec77f2cb61f2eba37237e42725923518b"}
Sep 14 07:57:31.261982 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.261815    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf" event={"ID":"e6e14278-6f9d-43f2-b812-59d052d46258","Type":"ContainerStarted","Data":"16d1818a51445fce345d68a028abcdbdb239c0dba4f0b1517ff63148a5f13b10"}
Sep 14 07:57:31.261982 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.261948    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:57:31.263330 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.263306    2556 generic.go:356] "Generic (PLEG): container finished" podID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" containerID="992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb" exitCode=0
Sep 14 07:57:31.263474 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.263353    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:31.263474 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.263366    2556 scope.go:117] "RemoveContainer" containerID="992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb"
Sep 14 07:57:31.271352 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.271331    2556 scope.go:117] "RemoveContainer" containerID="992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb"
Sep 14 07:57:31.271656 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:31.271635    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb\": container with ID starting with 992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb not found: ID does not exist" containerID="992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb"
Sep 14 07:57:31.271741 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.271666    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb"} err="failed to get container status \"992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb\": rpc error: code = NotFound desc = could not find container \"992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb\": container with ID starting with 992a8088a5dd403dd8b8249ec651bfa3e344ce73deb522c59391da90f8f3bccb not found: ID does not exist"
Sep 14 07:57:31.283596 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:31.283569    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sh8zf\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:31.283819 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:31.283784    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf" podStartSLOduration=1.2837720639999999 podStartE2EDuration="1.283772064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:31.28174484 +0000 UTC m=+507.781137652" watchObservedRunningTime="2026-09-14 07:57:31.283772064 +0000 UTC m=+507.783164901"
Sep 14 07:57:31.285194 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:57:31.285172    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sh8zf\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sh8zf"
Sep 14 07:57:32.018575 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:32.018538    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ffdaeae-caac-428e-bdfe-f2fd8618166a" path="/var/lib/kubelet/pods/6ffdaeae-caac-428e-bdfe-f2fd8618166a/volumes"
Sep 14 07:57:42.272108 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:57:42.272071    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xkkbf"
Sep 14 07:58:14.394236 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.394200    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:14.414000 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.413951    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:14.414157 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.414044    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.416294 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.416273    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 07:58:14.485925 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.485894    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:14.494303 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.494269    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv6db\" (UniqueName: \"kubernetes.io/projected/6136ceaa-80e4-4702-b0e9-7b08cb389be5-kube-api-access-sv6db\") pod \"limitador-limitador-8588749977-pxv6r\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") " pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.494445 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.494323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6136ceaa-80e4-4702-b0e9-7b08cb389be5-config-file\") pod \"limitador-limitador-8588749977-pxv6r\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") " pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.594744 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.594709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6136ceaa-80e4-4702-b0e9-7b08cb389be5-config-file\") pod \"limitador-limitador-8588749977-pxv6r\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") " pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.594893 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.594777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv6db\" (UniqueName: \"kubernetes.io/projected/6136ceaa-80e4-4702-b0e9-7b08cb389be5-kube-api-access-sv6db\") pod \"limitador-limitador-8588749977-pxv6r\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") " pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.595334 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.595313    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6136ceaa-80e4-4702-b0e9-7b08cb389be5-config-file\") pod \"limitador-limitador-8588749977-pxv6r\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") " pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.604531 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.604506    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv6db\" (UniqueName: \"kubernetes.io/projected/6136ceaa-80e4-4702-b0e9-7b08cb389be5-kube-api-access-sv6db\") pod \"limitador-limitador-8588749977-pxv6r\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") " pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.724564 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.724529    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:14.994776 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:14.994703    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:14.998181 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:58:14.998152    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6136ceaa_80e4_4702_b0e9_7b08cb389be5.slice/crio-77cb1089670749b158dc59ea4e911e9e622527aea2f5a37c88e773aba5fd286e WatchSource:0}: Error finding container 77cb1089670749b158dc59ea4e911e9e622527aea2f5a37c88e773aba5fd286e: Status 404 returned error can't find the container with id 77cb1089670749b158dc59ea4e911e9e622527aea2f5a37c88e773aba5fd286e
Sep 14 07:58:15.478499 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:15.478468    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r" event={"ID":"6136ceaa-80e4-4702-b0e9-7b08cb389be5","Type":"ContainerStarted","Data":"77cb1089670749b158dc59ea4e911e9e622527aea2f5a37c88e773aba5fd286e"}
Sep 14 07:58:17.488361 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:17.488324    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r" event={"ID":"6136ceaa-80e4-4702-b0e9-7b08cb389be5","Type":"ContainerStarted","Data":"8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e"}
Sep 14 07:58:17.488743 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:17.488393    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:17.511843 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:17.507583    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r" podStartSLOduration=1.428353341 podStartE2EDuration="3.507562819s" totalImagesPullingTime="2.079209478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:17.50520166 +0000 UTC m=+554.004594438" watchObservedRunningTime="2026-09-14 07:58:17.507562819 +0000 UTC m=+554.006955598"
Sep 14 07:58:28.243209 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.243168    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:28.243716 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.243386    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r" podUID="6136ceaa-80e4-4702-b0e9-7b08cb389be5" containerName="limitador" containerID="cri-o://8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e" gracePeriod=30
Sep 14 07:58:28.244342 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.244291    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:28.793590 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.793564    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:28.909986 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.909883    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6136ceaa-80e4-4702-b0e9-7b08cb389be5-config-file\") pod \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") "
Sep 14 07:58:28.910128 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.909998    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv6db\" (UniqueName: \"kubernetes.io/projected/6136ceaa-80e4-4702-b0e9-7b08cb389be5-kube-api-access-sv6db\") pod \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\" (UID: \"6136ceaa-80e4-4702-b0e9-7b08cb389be5\") "
Sep 14 07:58:28.910353 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.910325    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6136ceaa-80e4-4702-b0e9-7b08cb389be5-config-file" (OuterVolumeSpecName: "config-file") pod "6136ceaa-80e4-4702-b0e9-7b08cb389be5" (UID: "6136ceaa-80e4-4702-b0e9-7b08cb389be5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:58:28.912094 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:28.912068    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6136ceaa-80e4-4702-b0e9-7b08cb389be5-kube-api-access-sv6db" (OuterVolumeSpecName: "kube-api-access-sv6db") pod "6136ceaa-80e4-4702-b0e9-7b08cb389be5" (UID: "6136ceaa-80e4-4702-b0e9-7b08cb389be5"). InnerVolumeSpecName "kube-api-access-sv6db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:58:29.010523 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.010486    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv6db\" (UniqueName: \"kubernetes.io/projected/6136ceaa-80e4-4702-b0e9-7b08cb389be5-kube-api-access-sv6db\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:58:29.010523 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.010518    2556 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6136ceaa-80e4-4702-b0e9-7b08cb389be5-config-file\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:58:29.547105 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.547065    2556 generic.go:356] "Generic (PLEG): container finished" podID="6136ceaa-80e4-4702-b0e9-7b08cb389be5" containerID="8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e" exitCode=0
Sep 14 07:58:29.547468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.547112    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r"
Sep 14 07:58:29.547468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.547144    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r" event={"ID":"6136ceaa-80e4-4702-b0e9-7b08cb389be5","Type":"ContainerDied","Data":"8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e"}
Sep 14 07:58:29.547468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.547200    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pxv6r" event={"ID":"6136ceaa-80e4-4702-b0e9-7b08cb389be5","Type":"ContainerDied","Data":"77cb1089670749b158dc59ea4e911e9e622527aea2f5a37c88e773aba5fd286e"}
Sep 14 07:58:29.547468 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.547224    2556 scope.go:117] "RemoveContainer" containerID="8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e"
Sep 14 07:58:29.556045 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.556024    2556 scope.go:117] "RemoveContainer" containerID="8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e"
Sep 14 07:58:29.556309 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:58:29.556290    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e\": container with ID starting with 8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e not found: ID does not exist" containerID="8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e"
Sep 14 07:58:29.556353 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.556319    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e"} err="failed to get container status \"8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e\": rpc error: code = NotFound desc = could not find container \"8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e\": container with ID starting with 8ebd26523d6f50d6fca728f8aa3d50d5d503159de8b5bdb5383975dd3701db8e not found: ID does not exist"
Sep 14 07:58:29.566881 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.566851    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:29.570814 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:29.570792    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pxv6r"]
Sep 14 07:58:30.018507 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:30.018464    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6136ceaa-80e4-4702-b0e9-7b08cb389be5" path="/var/lib/kubelet/pods/6136ceaa-80e4-4702-b0e9-7b08cb389be5/volumes"
Sep 14 07:58:33.611502 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.611469    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"]
Sep 14 07:58:33.611944 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.611753    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6136ceaa-80e4-4702-b0e9-7b08cb389be5" containerName="limitador"
Sep 14 07:58:33.611944 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.611763    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6136ceaa-80e4-4702-b0e9-7b08cb389be5" containerName="limitador"
Sep 14 07:58:33.611944 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.611824    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6136ceaa-80e4-4702-b0e9-7b08cb389be5" containerName="limitador"
Sep 14 07:58:33.616117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.616097    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.618617 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.618593    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gxj2h\""
Sep 14 07:58:33.618730 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.618592    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 14 07:58:33.619304 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.619286    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 07:58:33.619400 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.619306    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 07:58:33.623495 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.623403    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"]
Sep 14 07:58:33.745303 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.745269    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8782a57-8e69-41b6-b0b4-fc1ebf949cbf-data\") pod \"postgres-69fb4594cf-8fsj7\" (UID: \"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.745451 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.745314    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scz2d\" (UniqueName: \"kubernetes.io/projected/c8782a57-8e69-41b6-b0b4-fc1ebf949cbf-kube-api-access-scz2d\") pod \"postgres-69fb4594cf-8fsj7\" (UID: \"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.845874 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.845841    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8782a57-8e69-41b6-b0b4-fc1ebf949cbf-data\") pod \"postgres-69fb4594cf-8fsj7\" (UID: \"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.846067 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.845887    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scz2d\" (UniqueName: \"kubernetes.io/projected/c8782a57-8e69-41b6-b0b4-fc1ebf949cbf-kube-api-access-scz2d\") pod \"postgres-69fb4594cf-8fsj7\" (UID: \"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.846243 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.846222    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8782a57-8e69-41b6-b0b4-fc1ebf949cbf-data\") pod \"postgres-69fb4594cf-8fsj7\" (UID: \"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.853807 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.853776    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scz2d\" (UniqueName: \"kubernetes.io/projected/c8782a57-8e69-41b6-b0b4-fc1ebf949cbf-kube-api-access-scz2d\") pod \"postgres-69fb4594cf-8fsj7\" (UID: \"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:33.927587 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:33.927564    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:34.198052 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:34.198027    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"]
Sep 14 07:58:34.200182 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:58:34.200146    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8782a57_8e69_41b6_b0b4_fc1ebf949cbf.slice/crio-0a0c1be17b7c0b956037fe4d97c71ca62af07c1c6d4d52738b4b34684695b15e WatchSource:0}: Error finding container 0a0c1be17b7c0b956037fe4d97c71ca62af07c1c6d4d52738b4b34684695b15e: Status 404 returned error can't find the container with id 0a0c1be17b7c0b956037fe4d97c71ca62af07c1c6d4d52738b4b34684695b15e
Sep 14 07:58:34.572685 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:34.572594    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7" event={"ID":"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf","Type":"ContainerStarted","Data":"0a0c1be17b7c0b956037fe4d97c71ca62af07c1c6d4d52738b4b34684695b15e"}
Sep 14 07:58:36.586318 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:36.586273    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7" event={"ID":"c8782a57-8e69-41b6-b0b4-fc1ebf949cbf","Type":"ContainerStarted","Data":"9f72968833fbe38815d124ff6649c5bc77f2c96f1ef2f3bcacc6bc431beb418e"}
Sep 14 07:58:36.586689 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:36.586346    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:36.602505 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:36.602428    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7" podStartSLOduration=1.459839092 podStartE2EDuration="3.602408841s" totalImagesPullingTime="2.142569749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:36.601947178 +0000 UTC m=+573.101339957" watchObservedRunningTime="2026-09-14 07:58:36.602408841 +0000 UTC m=+573.101801623"
Sep 14 07:58:42.618740 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:42.618669    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8fsj7"
Sep 14 07:58:52.117699 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.117658    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66cb75bc85-gcg6n"]
Sep 14 07:58:52.126803 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.126773    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.126974 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.126908    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66cb75bc85-gcg6n"]
Sep 14 07:58:52.129388 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.129359    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 07:58:52.129506 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.129369    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 07:58:52.129506 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.129404    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 07:58:52.129506 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.129432    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lrt2\""
Sep 14 07:58:52.288239 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.288208    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8944c479-b5jvw"]
Sep 14 07:58:52.291512 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.291491    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.298256 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.298233    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.298353 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.298266    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb4cp\" (UniqueName: \"kubernetes.io/projected/bfade098-ceda-46dc-9426-55f94540ae18-kube-api-access-tb4cp\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.298353 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.298341    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-metrics-certs\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.302486 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.302463    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8944c479-b5jvw"]
Sep 14 07:58:52.399355 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.399271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.399355 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.399308    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb4cp\" (UniqueName: \"kubernetes.io/projected/bfade098-ceda-46dc-9426-55f94540ae18-kube-api-access-tb4cp\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.399355 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.399342    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.399566 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.399359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-metrics-certs\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.399566 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.399389    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26rkn\" (UniqueName: \"kubernetes.io/projected/3627632d-b644-4597-8148-cd18640973ff-kube-api-access-26rkn\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.399566 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.399426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-metrics-certs\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.399566 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:58:52.399441    2556 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 14 07:58:52.399566 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:58:52.399522    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert podName:bfade098-ceda-46dc-9426-55f94540ae18 nodeName:}" failed. No retries permitted until 2026-09-14 07:58:52.899502322 +0000 UTC m=+589.398895093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert") pod "maas-controller-66cb75bc85-gcg6n" (UID: "bfade098-ceda-46dc-9426-55f94540ae18") : secret "maas-controller-webhook-cert" not found
Sep 14 07:58:52.401764 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.401741    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-metrics-certs\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.407194 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.407172    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb4cp\" (UniqueName: \"kubernetes.io/projected/bfade098-ceda-46dc-9426-55f94540ae18-kube-api-access-tb4cp\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.500304 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.500272    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.500471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.500312    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-metrics-certs\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.500471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.500350    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26rkn\" (UniqueName: \"kubernetes.io/projected/3627632d-b644-4597-8148-cd18640973ff-kube-api-access-26rkn\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.500471 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:58:52.500449    2556 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 14 07:58:52.500640 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:58:52.500510    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert podName:3627632d-b644-4597-8148-cd18640973ff nodeName:}" failed. No retries permitted until 2026-09-14 07:58:53.000492654 +0000 UTC m=+589.499885415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert") pod "maas-controller-6c8944c479-b5jvw" (UID: "3627632d-b644-4597-8148-cd18640973ff") : secret "maas-controller-webhook-cert" not found
Sep 14 07:58:52.502642 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.502621    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-metrics-certs\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.510436 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.510417    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26rkn\" (UniqueName: \"kubernetes.io/projected/3627632d-b644-4597-8148-cd18640973ff-kube-api-access-26rkn\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:52.904387 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.904345    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:52.906766 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:52.906739    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert\") pod \"maas-controller-66cb75bc85-gcg6n\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") " pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:53.004913 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.004878    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:53.007191 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.007171    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert\") pod \"maas-controller-6c8944c479-b5jvw\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") " pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:53.038541 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.038515    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:53.201872 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.201844    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:53.308401 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.308373    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66cb75bc85-gcg6n"]
Sep 14 07:58:53.309813 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:58:53.309781    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfade098_ceda_46dc_9426_55f94540ae18.slice/crio-a0e20f14eff255ebf8e537a104a9b6da634ba74e51279aff8aaabb154e6ada66 WatchSource:0}: Error finding container a0e20f14eff255ebf8e537a104a9b6da634ba74e51279aff8aaabb154e6ada66: Status 404 returned error can't find the container with id a0e20f14eff255ebf8e537a104a9b6da634ba74e51279aff8aaabb154e6ada66
Sep 14 07:58:53.475285 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.475260    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8944c479-b5jvw"]
Sep 14 07:58:53.477387 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:58:53.477358    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3627632d_b644_4597_8148_cd18640973ff.slice/crio-2d93c8387b31e82ffb8e2c0f1d723c6db2dfa9f5a1e9ec29c03ea435df71a4b2 WatchSource:0}: Error finding container 2d93c8387b31e82ffb8e2c0f1d723c6db2dfa9f5a1e9ec29c03ea435df71a4b2: Status 404 returned error can't find the container with id 2d93c8387b31e82ffb8e2c0f1d723c6db2dfa9f5a1e9ec29c03ea435df71a4b2
Sep 14 07:58:53.667823 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.667789    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-b5jvw" event={"ID":"3627632d-b644-4597-8148-cd18640973ff","Type":"ContainerStarted","Data":"2d93c8387b31e82ffb8e2c0f1d723c6db2dfa9f5a1e9ec29c03ea435df71a4b2"}
Sep 14 07:58:53.668869 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:53.668845    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66cb75bc85-gcg6n" event={"ID":"bfade098-ceda-46dc-9426-55f94540ae18","Type":"ContainerStarted","Data":"a0e20f14eff255ebf8e537a104a9b6da634ba74e51279aff8aaabb154e6ada66"}
Sep 14 07:58:56.689418 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:56.689370    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-b5jvw" event={"ID":"3627632d-b644-4597-8148-cd18640973ff","Type":"ContainerStarted","Data":"1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390"}
Sep 14 07:58:56.689878 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:56.689485    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:58:56.691460 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:56.691432    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66cb75bc85-gcg6n" event={"ID":"bfade098-ceda-46dc-9426-55f94540ae18","Type":"ContainerStarted","Data":"393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16"}
Sep 14 07:58:56.691568 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:56.691487    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:58:56.707729 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:56.707678    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8944c479-b5jvw" podStartSLOduration=1.885861454 podStartE2EDuration="4.707663398s" totalImagesPullingTime="2.821801944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:56.70605178 +0000 UTC m=+593.205444579" watchObservedRunningTime="2026-09-14 07:58:56.707663398 +0000 UTC m=+593.207056177"
Sep 14 07:58:56.724286 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:58:56.724243    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66cb75bc85-gcg6n" podStartSLOduration=1.727437852 podStartE2EDuration="4.724226625s" totalImagesPullingTime="2.996788773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:56.722627236 +0000 UTC m=+593.222020028" watchObservedRunningTime="2026-09-14 07:58:56.724226625 +0000 UTC m=+593.223619403"
Sep 14 07:59:07.701534 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:07.701501    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 07:59:07.701989 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:07.701581    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:59:07.762319 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:07.762268    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66cb75bc85-gcg6n"]
Sep 14 07:59:07.762632 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:07.762606    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66cb75bc85-gcg6n" podUID="bfade098-ceda-46dc-9426-55f94540ae18" containerName="manager" containerID="cri-o://393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16" gracePeriod=30
Sep 14 07:59:08.014049 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.014019    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:59:08.129733 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.129695    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb4cp\" (UniqueName: \"kubernetes.io/projected/bfade098-ceda-46dc-9426-55f94540ae18-kube-api-access-tb4cp\") pod \"bfade098-ceda-46dc-9426-55f94540ae18\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") "
Sep 14 07:59:08.129924 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.129753    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-metrics-certs\") pod \"bfade098-ceda-46dc-9426-55f94540ae18\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") "
Sep 14 07:59:08.129924 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.129779    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert\") pod \"bfade098-ceda-46dc-9426-55f94540ae18\" (UID: \"bfade098-ceda-46dc-9426-55f94540ae18\") "
Sep 14 07:59:08.132020 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.131980    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bfade098-ceda-46dc-9426-55f94540ae18" (UID: "bfade098-ceda-46dc-9426-55f94540ae18"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:59:08.132128 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.132042    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfade098-ceda-46dc-9426-55f94540ae18-kube-api-access-tb4cp" (OuterVolumeSpecName: "kube-api-access-tb4cp") pod "bfade098-ceda-46dc-9426-55f94540ae18" (UID: "bfade098-ceda-46dc-9426-55f94540ae18"). InnerVolumeSpecName "kube-api-access-tb4cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:59:08.132128 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.132078    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bfade098-ceda-46dc-9426-55f94540ae18" (UID: "bfade098-ceda-46dc-9426-55f94540ae18"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:59:08.231239 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.231196    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:59:08.231239 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.231234    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bfade098-ceda-46dc-9426-55f94540ae18-webhook-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:59:08.231239 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.231245    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb4cp\" (UniqueName: \"kubernetes.io/projected/bfade098-ceda-46dc-9426-55f94540ae18-kube-api-access-tb4cp\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 07:59:08.755655 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.755621    2556 generic.go:356] "Generic (PLEG): container finished" podID="bfade098-ceda-46dc-9426-55f94540ae18" containerID="393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16" exitCode=0
Sep 14 07:59:08.756162 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.755661    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66cb75bc85-gcg6n"
Sep 14 07:59:08.756162 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.755697    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66cb75bc85-gcg6n" event={"ID":"bfade098-ceda-46dc-9426-55f94540ae18","Type":"ContainerDied","Data":"393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16"}
Sep 14 07:59:08.756162 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.755750    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66cb75bc85-gcg6n" event={"ID":"bfade098-ceda-46dc-9426-55f94540ae18","Type":"ContainerDied","Data":"a0e20f14eff255ebf8e537a104a9b6da634ba74e51279aff8aaabb154e6ada66"}
Sep 14 07:59:08.756162 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.755782    2556 scope.go:117] "RemoveContainer" containerID="393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16"
Sep 14 07:59:08.765206 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.765019    2556 scope.go:117] "RemoveContainer" containerID="393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16"
Sep 14 07:59:08.765322 ip-10-0-131-236 kubenswrapper[2556]: E0914 07:59:08.765303    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16\": container with ID starting with 393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16 not found: ID does not exist" containerID="393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16"
Sep 14 07:59:08.765363 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.765331    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16"} err="failed to get container status \"393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16\": rpc error: code = NotFound desc = could not find container \"393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16\": container with ID starting with 393dc088443f32d987ae55097895ad1d175694bead67b08e0e75ca921329eb16 not found: ID does not exist"
Sep 14 07:59:08.776005 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.775972    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66cb75bc85-gcg6n"]
Sep 14 07:59:08.778471 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:08.778451    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66cb75bc85-gcg6n"]
Sep 14 07:59:10.018497 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:10.018466    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfade098-ceda-46dc-9426-55f94540ae18" path="/var/lib/kubelet/pods/bfade098-ceda-46dc-9426-55f94540ae18/volumes"
Sep 14 07:59:28.176954 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.176878    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"]
Sep 14 07:59:28.177308 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.177201    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfade098-ceda-46dc-9426-55f94540ae18" containerName="manager"
Sep 14 07:59:28.177308 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.177213    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfade098-ceda-46dc-9426-55f94540ae18" containerName="manager"
Sep 14 07:59:28.177308 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.177277    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfade098-ceda-46dc-9426-55f94540ae18" containerName="manager"
Sep 14 07:59:28.180012 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.179994    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.181983 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.181942    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 07:59:28.182117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.181943    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 07:59:28.182117 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.181993    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 07:59:28.189411 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.189388    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"]
Sep 14 07:59:28.292748 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.292714    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-maas-api-tls\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.292945 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.292761    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-openshift-service-ca\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.292945 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.292835    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-kube-api-access-t7qsg\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.292945 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.292878    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-metrics-certs\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.393596 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.393558    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-openshift-service-ca\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.393766 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.393634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-kube-api-access-t7qsg\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.393766 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.393666    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-metrics-certs\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.393766 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.393738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-maas-api-tls\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.394269 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.394247    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-openshift-service-ca\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.396318 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.396294    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-metrics-certs\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.396318 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.396303    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-maas-api-tls\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.400543 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.400524    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-kube-api-access-t7qsg\") pod \"maas-api-7f5c8b48b7-jcvhj\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") " pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.491656 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.491575    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:28.624533 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.624502    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"]
Sep 14 07:59:28.629551 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.629526    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.631617 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.631591    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 07:59:28.631741 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.631642    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 07:59:28.631741 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.631657    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 07:59:28.639099 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.639076    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"]
Sep 14 07:59:28.669412 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.669372    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"]
Sep 14 07:59:28.672755 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.672735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.674842 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.674825    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gjgpn\""
Sep 14 07:59:28.683236 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.683214    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"]
Sep 14 07:59:28.696061 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.696034    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86a2a63-e727-45a1-bb6b-4e11e821e409-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gnjdc\" (UID: \"e86a2a63-e727-45a1-bb6b-4e11e821e409\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.696178 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.696130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59xlb\" (UniqueName: \"kubernetes.io/projected/e86a2a63-e727-45a1-bb6b-4e11e821e409-kube-api-access-59xlb\") pod \"payload-pre-processing-59fbc47bf8-gnjdc\" (UID: \"e86a2a63-e727-45a1-bb6b-4e11e821e409\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.767679 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.767655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"]
Sep 14 07:59:28.770519 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:59:28.770495    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ab9891_a870_4b86_a3cd_2bfb7a5b674f.slice/crio-20374b6a8f0ff6837847cbab8bc543b35b272ad7b15bef97530672aa16bb76a8 WatchSource:0}: Error finding container 20374b6a8f0ff6837847cbab8bc543b35b272ad7b15bef97530672aa16bb76a8: Status 404 returned error can't find the container with id 20374b6a8f0ff6837847cbab8bc543b35b272ad7b15bef97530672aa16bb76a8
Sep 14 07:59:28.772447 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.772427    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:59:28.797197 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.797169    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kh2b\" (UniqueName: \"kubernetes.io/projected/1231c9fa-d159-43fd-8c3a-024a981babde-kube-api-access-2kh2b\") pod \"payload-processing-5b54b7fbb7-q6hxd\" (UID: \"1231c9fa-d159-43fd-8c3a-024a981babde\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.797291 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.797216    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59xlb\" (UniqueName: \"kubernetes.io/projected/e86a2a63-e727-45a1-bb6b-4e11e821e409-kube-api-access-59xlb\") pod \"payload-pre-processing-59fbc47bf8-gnjdc\" (UID: \"e86a2a63-e727-45a1-bb6b-4e11e821e409\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.797291 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.797242    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1231c9fa-d159-43fd-8c3a-024a981babde-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-q6hxd\" (UID: \"1231c9fa-d159-43fd-8c3a-024a981babde\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.797291 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.797287    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86a2a63-e727-45a1-bb6b-4e11e821e409-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gnjdc\" (UID: \"e86a2a63-e727-45a1-bb6b-4e11e821e409\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.797930 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.797910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86a2a63-e727-45a1-bb6b-4e11e821e409-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gnjdc\" (UID: \"e86a2a63-e727-45a1-bb6b-4e11e821e409\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.804470 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.804451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59xlb\" (UniqueName: \"kubernetes.io/projected/e86a2a63-e727-45a1-bb6b-4e11e821e409-kube-api-access-59xlb\") pod \"payload-pre-processing-59fbc47bf8-gnjdc\" (UID: \"e86a2a63-e727-45a1-bb6b-4e11e821e409\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.858090 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.858057    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj" event={"ID":"55ab9891-a870-4b86-a3cd-2bfb7a5b674f","Type":"ContainerStarted","Data":"20374b6a8f0ff6837847cbab8bc543b35b272ad7b15bef97530672aa16bb76a8"}
Sep 14 07:59:28.897678 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.897649    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kh2b\" (UniqueName: \"kubernetes.io/projected/1231c9fa-d159-43fd-8c3a-024a981babde-kube-api-access-2kh2b\") pod \"payload-processing-5b54b7fbb7-q6hxd\" (UID: \"1231c9fa-d159-43fd-8c3a-024a981babde\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.897834 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.897703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1231c9fa-d159-43fd-8c3a-024a981babde-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-q6hxd\" (UID: \"1231c9fa-d159-43fd-8c3a-024a981babde\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.898464 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.898444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1231c9fa-d159-43fd-8c3a-024a981babde-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-q6hxd\" (UID: \"1231c9fa-d159-43fd-8c3a-024a981babde\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.904716 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.904688    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kh2b\" (UniqueName: \"kubernetes.io/projected/1231c9fa-d159-43fd-8c3a-024a981babde-kube-api-access-2kh2b\") pod \"payload-processing-5b54b7fbb7-q6hxd\" (UID: \"1231c9fa-d159-43fd-8c3a-024a981babde\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:28.941555 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.941531    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:28.982926 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:28.982894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:29.213258 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:29.213231    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"]
Sep 14 07:59:29.214976 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:59:29.214937    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86a2a63_e727_45a1_bb6b_4e11e821e409.slice/crio-6d3d270785148f993afa457722b198bd10a5a75600d73e525207235742690d6c WatchSource:0}: Error finding container 6d3d270785148f993afa457722b198bd10a5a75600d73e525207235742690d6c: Status 404 returned error can't find the container with id 6d3d270785148f993afa457722b198bd10a5a75600d73e525207235742690d6c
Sep 14 07:59:29.259670 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:29.259646    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"]
Sep 14 07:59:29.261603 ip-10-0-131-236 kubenswrapper[2556]: W0914 07:59:29.261571    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1231c9fa_d159_43fd_8c3a_024a981babde.slice/crio-60a7c9bab2b5984a67c79e084a9a09b0d23a34aaa13c1824ed111cabc4dae0fa WatchSource:0}: Error finding container 60a7c9bab2b5984a67c79e084a9a09b0d23a34aaa13c1824ed111cabc4dae0fa: Status 404 returned error can't find the container with id 60a7c9bab2b5984a67c79e084a9a09b0d23a34aaa13c1824ed111cabc4dae0fa
Sep 14 07:59:29.867984 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:29.867926    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc" event={"ID":"e86a2a63-e727-45a1-bb6b-4e11e821e409","Type":"ContainerStarted","Data":"6d3d270785148f993afa457722b198bd10a5a75600d73e525207235742690d6c"}
Sep 14 07:59:29.869312 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:29.869282    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd" event={"ID":"1231c9fa-d159-43fd-8c3a-024a981babde","Type":"ContainerStarted","Data":"60a7c9bab2b5984a67c79e084a9a09b0d23a34aaa13c1824ed111cabc4dae0fa"}
Sep 14 07:59:30.877220 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.877183    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd" event={"ID":"1231c9fa-d159-43fd-8c3a-024a981babde","Type":"ContainerStarted","Data":"5195d8c5f2c762b55dfc991fafdf4605068f04cd7f4a73af079ad7bdd359e801"}
Sep 14 07:59:30.877733 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.877270    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:30.879648 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.879548    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj" event={"ID":"55ab9891-a870-4b86-a3cd-2bfb7a5b674f","Type":"ContainerStarted","Data":"4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b"}
Sep 14 07:59:30.879648 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.879623    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:30.881478 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.881454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc" event={"ID":"e86a2a63-e727-45a1-bb6b-4e11e821e409","Type":"ContainerStarted","Data":"59add02169f080a780a66a0fb867edb8f91284d6f6073c71c65548ce94bdbbda"}
Sep 14 07:59:30.881869 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.881842    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 07:59:30.892189 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.892115    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd" podStartSLOduration=1.369771962 podStartE2EDuration="2.89209821s" totalImagesPullingTime="1.522326248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:30.89134457 +0000 UTC m=+627.390737374" watchObservedRunningTime="2026-09-14 07:59:30.89209821 +0000 UTC m=+627.391490990"
Sep 14 07:59:30.907917 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.907870    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc" podStartSLOduration=1.338077729 podStartE2EDuration="2.907857123s" totalImagesPullingTime="1.569779394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:30.907557379 +0000 UTC m=+627.406950183" watchObservedRunningTime="2026-09-14 07:59:30.907857123 +0000 UTC m=+627.407249901"
Sep 14 07:59:30.923544 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:30.923476    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj" podStartSLOduration=0.910071046 podStartE2EDuration="2.923461935s" totalImagesPullingTime="2.013390889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:30.921325641 +0000 UTC m=+627.420718446" watchObservedRunningTime="2026-09-14 07:59:30.923461935 +0000 UTC m=+627.422854713"
Sep 14 07:59:36.892930 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:36.892901    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 07:59:41.888699 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:41.888664    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-q6hxd"
Sep 14 07:59:41.889311 ip-10-0-131-236 kubenswrapper[2556]: I0914 07:59:41.889020    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gnjdc"
Sep 14 08:00:05.775183 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.775146    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"]
Sep 14 08:00:05.778555 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.778534    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.781321 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.781296    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 08:00:05.781321 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.781318    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 08:00:05.781532 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.781336    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 08:00:05.781532 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.781426    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dsj5g\""
Sep 14 08:00:05.785534 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.785478    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"]
Sep 14 08:00:05.897104 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.897073    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.897262 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.897116    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.897262 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.897171    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zckc\" (UniqueName: \"kubernetes.io/projected/a170840a-585e-42a6-9789-d8478b72f676-kube-api-access-7zckc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.897262 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.897209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.897262 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.897230    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a170840a-585e-42a6-9789-d8478b72f676-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.897262 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.897246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998483 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998659 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998659 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zckc\" (UniqueName: \"kubernetes.io/projected/a170840a-585e-42a6-9789-d8478b72f676-kube-api-access-7zckc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998659 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998584    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998659 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a170840a-585e-42a6-9789-d8478b72f676-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998659 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998635    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.998904 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.998885    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.999026 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.999006    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:05.999111 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:05.999086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:06.000710 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:06.000689    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a170840a-585e-42a6-9789-d8478b72f676-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:06.000885 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:06.000866    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a170840a-585e-42a6-9789-d8478b72f676-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:06.005340 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:06.005319    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zckc\" (UniqueName: \"kubernetes.io/projected/a170840a-585e-42a6-9789-d8478b72f676-kube-api-access-7zckc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-286hc\" (UID: \"a170840a-585e-42a6-9789-d8478b72f676\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:06.091122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:06.091056    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:06.379569 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:06.379536    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"]
Sep 14 08:00:06.381405 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:00:06.381371    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda170840a_585e_42a6_9789_d8478b72f676.slice/crio-b732467c48e709ceb6422e40659db614ea9472484b1e0fcf3199f3816470df70 WatchSource:0}: Error finding container b732467c48e709ceb6422e40659db614ea9472484b1e0fcf3199f3816470df70: Status 404 returned error can't find the container with id b732467c48e709ceb6422e40659db614ea9472484b1e0fcf3199f3816470df70
Sep 14 08:00:07.065746 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:07.065683    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc" event={"ID":"a170840a-585e-42a6-9789-d8478b72f676","Type":"ContainerStarted","Data":"b732467c48e709ceb6422e40659db614ea9472484b1e0fcf3199f3816470df70"}
Sep 14 08:00:08.071778 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:08.071737    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc" event={"ID":"a170840a-585e-42a6-9789-d8478b72f676","Type":"ContainerStarted","Data":"8bfe30785eddb71b1365e7889a5148d3d84668cca89424ff920bbf16b20d31f2"}
Sep 14 08:00:08.088701 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:08.088656    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc" podStartSLOduration=1.812104081 podStartE2EDuration="3.088642298s" totalImagesPullingTime="1.276538217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:08.08595584 +0000 UTC m=+664.585348623" watchObservedRunningTime="2026-09-14 08:00:08.088642298 +0000 UTC m=+664.588035076"
Sep 14 08:00:09.579388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.579348    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"]
Sep 14 08:00:09.581718 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.581701    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.583935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.583915    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 14 08:00:09.590684 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.590657    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"]
Sep 14 08:00:09.628492 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.628448    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jcsf\" (UniqueName: \"kubernetes.io/projected/613e05d0-bc03-4f82-8375-f4c77f1b72e9-kube-api-access-8jcsf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.628492 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.628496    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.628687 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.628551    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.628687 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.628604    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613e05d0-bc03-4f82-8375-f4c77f1b72e9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.628687 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.628633    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.628687 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.628683    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729245 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jcsf\" (UniqueName: \"kubernetes.io/projected/613e05d0-bc03-4f82-8375-f4c77f1b72e9-kube-api-access-8jcsf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729245 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729137    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729245 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729167    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729245 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729193    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613e05d0-bc03-4f82-8375-f4c77f1b72e9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729426 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729269    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729537 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729537 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729517    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.729695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.729571    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.731398 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.731378    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613e05d0-bc03-4f82-8375-f4c77f1b72e9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.731741 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.731722    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613e05d0-bc03-4f82-8375-f4c77f1b72e9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.735842 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.735816    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jcsf\" (UniqueName: \"kubernetes.io/projected/613e05d0-bc03-4f82-8375-f4c77f1b72e9-kube-api-access-8jcsf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4\" (UID: \"613e05d0-bc03-4f82-8375-f4c77f1b72e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:09.892309 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:09.892211    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:10.162654 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:10.162629    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"]
Sep 14 08:00:10.164734 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:00:10.164711    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613e05d0_bc03_4f82_8375_f4c77f1b72e9.slice/crio-956fc47b29b9c80ed6374dbf7c426eb286a5430ade4b410d8803f853d3c0ce95 WatchSource:0}: Error finding container 956fc47b29b9c80ed6374dbf7c426eb286a5430ade4b410d8803f853d3c0ce95: Status 404 returned error can't find the container with id 956fc47b29b9c80ed6374dbf7c426eb286a5430ade4b410d8803f853d3c0ce95
Sep 14 08:00:11.090444 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:11.090402    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4" event={"ID":"613e05d0-bc03-4f82-8375-f4c77f1b72e9","Type":"ContainerStarted","Data":"c7eb66082de11b0ddd50dce6364413264c9eb648d6bbd253203636415a23e785"}
Sep 14 08:00:11.090820 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:11.090459    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4" event={"ID":"613e05d0-bc03-4f82-8375-f4c77f1b72e9","Type":"ContainerStarted","Data":"956fc47b29b9c80ed6374dbf7c426eb286a5430ade4b410d8803f853d3c0ce95"}
Sep 14 08:00:11.106951 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:11.106898    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4" podStartSLOduration=1.84880726 podStartE2EDuration="2.106883029s" totalImagesPullingTime="258.075769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:11.106373913 +0000 UTC m=+667.605766728" watchObservedRunningTime="2026-09-14 08:00:11.106883029 +0000 UTC m=+667.606275806"
Sep 14 08:00:16.092194 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:16.092155    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:16.092691 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:16.092216    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:16.104744 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:16.104717    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:16.127434 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:16.127399    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-286hc"
Sep 14 08:00:19.892786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:19.892742    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:19.893202 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:19.892822    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:19.909520 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:19.909489    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:00:20.146232 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:00:20.146156    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4"
Sep 14 08:02:52.720451 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.720408    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"]
Sep 14 08:02:52.723462 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.723442    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.725800 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.725778    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e7b0a8-serving-cert\""
Sep 14 08:02:52.734374 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.734347    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"]
Sep 14 08:02:52.812767 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.812729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-metrics-certs\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.812767 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.812765    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvzm\" (UniqueName: \"kubernetes.io/projected/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-kube-api-access-tvvzm\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.812992 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.812851    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.812992 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.812889    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.913324 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.913278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.913324 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.913328    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.913593 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.913370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-metrics-certs\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.913593 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.913389    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvzm\" (UniqueName: \"kubernetes.io/projected/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-kube-api-access-tvvzm\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.913593 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:02:52.913455    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-serving-cert: secret "maas-api-e2e-inf-a-e7b0a8-serving-cert" not found
Sep 14 08:02:52.913593 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:02:52.913543    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls podName:afec4335-6dc4-4f6e-bd54-f1a0a00ff253 nodeName:}" failed. No retries permitted until 2026-09-14 08:02:53.413522453 +0000 UTC m=+829.912915214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls") pod "maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" (UID: "afec4335-6dc4-4f6e-bd54-f1a0a00ff253") : secret "maas-api-e2e-inf-a-e7b0a8-serving-cert" not found
Sep 14 08:02:52.914205 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.914183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.915902 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.915880    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-metrics-certs\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:52.931190 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:52.931165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvzm\" (UniqueName: \"kubernetes.io/projected/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-kube-api-access-tvvzm\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:53.419316 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.419285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:53.421609 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.421582    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:53.474344 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.474314    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"]
Sep 14 08:02:53.476819 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.476799    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.479266 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.479243    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e7b0a8-dockercfg-vvn27\""
Sep 14 08:02:53.479373 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.479243    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e7b0a8\""
Sep 14 08:02:53.489763 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.489732    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"]
Sep 14 08:02:53.520452 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.520424    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqz2r\" (UniqueName: \"kubernetes.io/projected/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-kube-api-access-wqz2r\") pod \"payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.520555 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.520456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.524778 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.524758    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"]
Sep 14 08:02:53.526912 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.526898    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.544269 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.544248    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"]
Sep 14 08:02:53.621119 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.621090    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kfbw\" (UniqueName: \"kubernetes.io/projected/18f10aff-5699-4147-aa25-7e9326a1757c-kube-api-access-9kfbw\") pod \"payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.621286 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.621147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqz2r\" (UniqueName: \"kubernetes.io/projected/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-kube-api-access-wqz2r\") pod \"payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.621286 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.621202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.621286 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.621237    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f10aff-5699-4147-aa25-7e9326a1757c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.621851 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.621833    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.629161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.629136    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqz2r\" (UniqueName: \"kubernetes.io/projected/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-kube-api-access-wqz2r\") pod \"payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.635941 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.635917    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:53.721698 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.721665    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kfbw\" (UniqueName: \"kubernetes.io/projected/18f10aff-5699-4147-aa25-7e9326a1757c-kube-api-access-9kfbw\") pod \"payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.722144 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.721768    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f10aff-5699-4147-aa25-7e9326a1757c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.722520 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.722501    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f10aff-5699-4147-aa25-7e9326a1757c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.735030 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.735005    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kfbw\" (UniqueName: \"kubernetes.io/projected/18f10aff-5699-4147-aa25-7e9326a1757c-kube-api-access-9kfbw\") pod \"payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.738825 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.738804    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"]
Sep 14 08:02:53.741662 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.741645    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.743866 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.743847    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-cd7582-serving-cert\""
Sep 14 08:02:53.757014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.756990    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"]
Sep 14 08:02:53.787192 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.787167    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:53.823554 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.822725    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2qv\" (UniqueName: \"kubernetes.io/projected/c417be04-938b-4acf-a7eb-91fe8e7fa091-kube-api-access-hk2qv\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.823554 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.822778    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.823554 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.822812    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-metrics-certs\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.823554 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.822864    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417be04-938b-4acf-a7eb-91fe8e7fa091-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.836145 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.836114    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:53.914766 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.914732    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"]
Sep 14 08:02:53.916852 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:02:53.916823    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafec4335_6dc4_4f6e_bd54_f1a0a00ff253.slice/crio-847ab0e33a54d1786ae68368d2d2818d8616d5decdbf0996cb62fc92d09de1ab WatchSource:0}: Error finding container 847ab0e33a54d1786ae68368d2d2818d8616d5decdbf0996cb62fc92d09de1ab: Status 404 returned error can't find the container with id 847ab0e33a54d1786ae68368d2d2818d8616d5decdbf0996cb62fc92d09de1ab
Sep 14 08:02:53.923553 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.923531    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2qv\" (UniqueName: \"kubernetes.io/projected/c417be04-938b-4acf-a7eb-91fe8e7fa091-kube-api-access-hk2qv\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.923655 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.923575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.923655 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.923611    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-metrics-certs\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.923759 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.923663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417be04-938b-4acf-a7eb-91fe8e7fa091-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.924517 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.924492    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417be04-938b-4acf-a7eb-91fe8e7fa091-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.926217 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.926183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-metrics-certs\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.926385 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.926363    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:53.931941 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:53.931910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2qv\" (UniqueName: \"kubernetes.io/projected/c417be04-938b-4acf-a7eb-91fe8e7fa091-kube-api-access-hk2qv\") pod \"maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:54.053929 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.053841    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:54.055684 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.055664    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"]
Sep 14 08:02:54.057634 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:02:54.057610    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f0bfc9_c7a8_4a7d_a4e4_3856165b9098.slice/crio-13b032d4ededb8a6dfb11c7aa820eaeb6e09c79ee741762072aad311c38d6d88 WatchSource:0}: Error finding container 13b032d4ededb8a6dfb11c7aa820eaeb6e09c79ee741762072aad311c38d6d88: Status 404 returned error can't find the container with id 13b032d4ededb8a6dfb11c7aa820eaeb6e09c79ee741762072aad311c38d6d88
Sep 14 08:02:54.121339 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.121310    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"]
Sep 14 08:02:54.125039 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:02:54.124945    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f10aff_5699_4147_aa25_7e9326a1757c.slice/crio-201f7d53d417223fc0aa7d8ad8ac1e610287ef2060405fac5ebd0dc37d12d17a WatchSource:0}: Error finding container 201f7d53d417223fc0aa7d8ad8ac1e610287ef2060405fac5ebd0dc37d12d17a: Status 404 returned error can't find the container with id 201f7d53d417223fc0aa7d8ad8ac1e610287ef2060405fac5ebd0dc37d12d17a
Sep 14 08:02:54.330199 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.330176    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"]
Sep 14 08:02:54.332147 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:02:54.332121    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc417be04_938b_4acf_a7eb_91fe8e7fa091.slice/crio-52105d29ebb1d933530b48d0be2ec4b884cee86fce58ef72fcbe7d44604ac073 WatchSource:0}: Error finding container 52105d29ebb1d933530b48d0be2ec4b884cee86fce58ef72fcbe7d44604ac073: Status 404 returned error can't find the container with id 52105d29ebb1d933530b48d0be2ec4b884cee86fce58ef72fcbe7d44604ac073
Sep 14 08:02:54.370847 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.370814    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"]
Sep 14 08:02:54.373307 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.373286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.375871 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.375850    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-cd7582\""
Sep 14 08:02:54.376057 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.376035    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-cd7582-dockercfg-t5vft\""
Sep 14 08:02:54.386503 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.386477    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"]
Sep 14 08:02:54.429238 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.429211    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615d2e12-74e3-47d9-876d-94a34276f5c3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.429357 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.429262    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txxrj\" (UniqueName: \"kubernetes.io/projected/615d2e12-74e3-47d9-876d-94a34276f5c3-kube-api-access-txxrj\") pod \"payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.530668 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.530631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615d2e12-74e3-47d9-876d-94a34276f5c3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.530854 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.530684    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txxrj\" (UniqueName: \"kubernetes.io/projected/615d2e12-74e3-47d9-876d-94a34276f5c3-kube-api-access-txxrj\") pod \"payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.531464 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.531438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615d2e12-74e3-47d9-876d-94a34276f5c3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.538159 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.538138    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txxrj\" (UniqueName: \"kubernetes.io/projected/615d2e12-74e3-47d9-876d-94a34276f5c3-kube-api-access-txxrj\") pod \"payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.685578 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.685520    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:54.915513 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.915229    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz" event={"ID":"18f10aff-5699-4147-aa25-7e9326a1757c","Type":"ContainerStarted","Data":"a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3"}
Sep 14 08:02:54.915513 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.915304    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz" event={"ID":"18f10aff-5699-4147-aa25-7e9326a1757c","Type":"ContainerStarted","Data":"201f7d53d417223fc0aa7d8ad8ac1e610287ef2060405fac5ebd0dc37d12d17a"}
Sep 14 08:02:54.916134 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.916077    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.918059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c" event={"ID":"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098","Type":"ContainerStarted","Data":"53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf"}
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.918113    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.918137    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c" event={"ID":"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098","Type":"ContainerStarted","Data":"13b032d4ededb8a6dfb11c7aa820eaeb6e09c79ee741762072aad311c38d6d88"}
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.920585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" event={"ID":"afec4335-6dc4-4f6e-bd54-f1a0a00ff253","Type":"ContainerStarted","Data":"5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8"}
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.920630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" event={"ID":"afec4335-6dc4-4f6e-bd54-f1a0a00ff253","Type":"ContainerStarted","Data":"847ab0e33a54d1786ae68368d2d2818d8616d5decdbf0996cb62fc92d09de1ab"}
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.921017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.922727    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4" event={"ID":"c417be04-938b-4acf-a7eb-91fe8e7fa091","Type":"ContainerStarted","Data":"2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf"}
Sep 14 08:02:54.923538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.922767    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4" event={"ID":"c417be04-938b-4acf-a7eb-91fe8e7fa091","Type":"ContainerStarted","Data":"52105d29ebb1d933530b48d0be2ec4b884cee86fce58ef72fcbe7d44604ac073"}
Sep 14 08:02:54.932026 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.925270    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:02:54.939016 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.938935    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz" podStartSLOduration=1.938914662 podStartE2EDuration="1.938914662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:54.932352188 +0000 UTC m=+831.431744967" watchObservedRunningTime="2026-09-14 08:02:54.938914662 +0000 UTC m=+831.438307445"
Sep 14 08:02:54.954871 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.954814    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" podStartSLOduration=2.477997688 podStartE2EDuration="2.954794677s" totalImagesPullingTime="476.796989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:54.952510829 +0000 UTC m=+831.451903628" watchObservedRunningTime="2026-09-14 08:02:54.954794677 +0000 UTC m=+831.454187463"
Sep 14 08:02:54.963328 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.963300    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"]
Sep 14 08:02:54.965086 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:02:54.965043    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615d2e12_74e3_47d9_876d_94a34276f5c3.slice/crio-4df17c33047ac4ae52aa19cbaf64046f19bf3afcb296520a677b614ecf71f188 WatchSource:0}: Error finding container 4df17c33047ac4ae52aa19cbaf64046f19bf3afcb296520a677b614ecf71f188: Status 404 returned error can't find the container with id 4df17c33047ac4ae52aa19cbaf64046f19bf3afcb296520a677b614ecf71f188
Sep 14 08:02:54.973698 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:54.973655    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4" podStartSLOduration=1.678212727 podStartE2EDuration="1.973641918s" totalImagesPullingTime="295.429191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:54.970347685 +0000 UTC m=+831.469740489" watchObservedRunningTime="2026-09-14 08:02:54.973641918 +0000 UTC m=+831.473034696"
Sep 14 08:02:55.008330 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.008276    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c" podStartSLOduration=2.008254413 podStartE2EDuration="2.008254413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:55.006521289 +0000 UTC m=+831.505914136" watchObservedRunningTime="2026-09-14 08:02:55.008254413 +0000 UTC m=+831.507647196"
Sep 14 08:02:55.591389 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.591352    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"]
Sep 14 08:02:55.593771 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.593754    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.596601 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.596416    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-831f8e-dockercfg-cb7b9\""
Sep 14 08:02:55.596791 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.596769    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-831f8e\""
Sep 14 08:02:55.606918 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.606887    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"]
Sep 14 08:02:55.642695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.642664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.642913 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.642886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4m8c\" (UniqueName: \"kubernetes.io/projected/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-kube-api-access-t4m8c\") pod \"payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.743897 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.743860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.744082 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.743910    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4m8c\" (UniqueName: \"kubernetes.io/projected/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-kube-api-access-t4m8c\") pod \"payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.744773 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.744749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.752592 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.752555    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4m8c\" (UniqueName: \"kubernetes.io/projected/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-kube-api-access-t4m8c\") pod \"payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.907368 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.907286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:55.931419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.931383    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n" event={"ID":"615d2e12-74e3-47d9-876d-94a34276f5c3","Type":"ContainerStarted","Data":"c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7"}
Sep 14 08:02:55.931795 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.931445    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n" event={"ID":"615d2e12-74e3-47d9-876d-94a34276f5c3","Type":"ContainerStarted","Data":"4df17c33047ac4ae52aa19cbaf64046f19bf3afcb296520a677b614ecf71f188"}
Sep 14 08:02:55.931795 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.931682    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:02:55.953850 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:55.953792    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n" podStartSLOduration=1.953772499 podStartE2EDuration="1.953772499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:55.950876696 +0000 UTC m=+832.450269497" watchObservedRunningTime="2026-09-14 08:02:55.953772499 +0000 UTC m=+832.453165279"
Sep 14 08:02:56.184685 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:56.184655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"]
Sep 14 08:02:56.186152 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:02:56.186127    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7fc9632_deb5_45ec_8d72_26aa1e42fc54.slice/crio-0ad3e5ae5ea9a7af2cc8bd32fa6b839645c2a1918ed0b8eefd5a320f088d52ee WatchSource:0}: Error finding container 0ad3e5ae5ea9a7af2cc8bd32fa6b839645c2a1918ed0b8eefd5a320f088d52ee: Status 404 returned error can't find the container with id 0ad3e5ae5ea9a7af2cc8bd32fa6b839645c2a1918ed0b8eefd5a320f088d52ee
Sep 14 08:02:56.940344 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:56.940289    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s" event={"ID":"c7fc9632-deb5-45ec-8d72-26aa1e42fc54","Type":"ContainerStarted","Data":"9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a"}
Sep 14 08:02:56.940788 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:56.940771    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:02:56.940855 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:56.940813    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s" event={"ID":"c7fc9632-deb5-45ec-8d72-26aa1e42fc54","Type":"ContainerStarted","Data":"0ad3e5ae5ea9a7af2cc8bd32fa6b839645c2a1918ed0b8eefd5a320f088d52ee"}
Sep 14 08:02:56.961324 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:02:56.961274    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s" podStartSLOduration=1.961255049 podStartE2EDuration="1.961255049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:56.957716121 +0000 UTC m=+833.457108919" watchObservedRunningTime="2026-09-14 08:02:56.961255049 +0000 UTC m=+833.460647826"
Sep 14 08:03:00.937121 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:00.937086    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:03:00.937631 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:00.937610    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:03:05.934562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:05.934519    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:03:06.942154 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:06.942120    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:03:06.942527 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:06.942195    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:03:07.135173 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.135134    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"]
Sep 14 08:03:07.138825 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.138804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.140811 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.140790    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-885430-serving-cert\""
Sep 14 08:03:07.149267 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.149243    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"]
Sep 14 08:03:07.245197 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.245105    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.245197 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.245181    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-metrics-certs\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.245406 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.245209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw5pq\" (UniqueName: \"kubernetes.io/projected/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-kube-api-access-mw5pq\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.245406 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.245273    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-openshift-service-ca\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.345840 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.345803    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.346033 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.345878    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-metrics-certs\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.346033 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.345904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5pq\" (UniqueName: \"kubernetes.io/projected/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-kube-api-access-mw5pq\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.346033 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.345935    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-openshift-service-ca\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.346033 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:03:07.346000    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-885430-serving-cert: secret "maas-api-e2e-autores-885430-serving-cert" not found
Sep 14 08:03:07.346183 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:03:07.346096    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls podName:2d21f285-f37f-4c3e-a9f3-9a3341381bf6 nodeName:}" failed. No retries permitted until 2026-09-14 08:03:07.846068053 +0000 UTC m=+844.345460823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls") pod "maas-api-e2e-autores-885430-5cb99546df-f6htw" (UID: "2d21f285-f37f-4c3e-a9f3-9a3341381bf6") : secret "maas-api-e2e-autores-885430-serving-cert" not found
Sep 14 08:03:07.346657 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.346635    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-openshift-service-ca\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.348386 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.348362    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-metrics-certs\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.354614 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.354580    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw5pq\" (UniqueName: \"kubernetes.io/projected/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-kube-api-access-mw5pq\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.850370 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.850328    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:07.850562 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:03:07.850493    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-885430-serving-cert: secret "maas-api-e2e-autores-885430-serving-cert" not found
Sep 14 08:03:07.850638 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:03:07.850605    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls podName:2d21f285-f37f-4c3e-a9f3-9a3341381bf6 nodeName:}" failed. No retries permitted until 2026-09-14 08:03:08.850581575 +0000 UTC m=+845.349974336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls") pod "maas-api-e2e-autores-885430-5cb99546df-f6htw" (UID: "2d21f285-f37f-4c3e-a9f3-9a3341381bf6") : secret "maas-api-e2e-autores-885430-serving-cert" not found
Sep 14 08:03:07.967151 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.967118    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"]
Sep 14 08:03:07.970904 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.970884    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:07.973108 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.973090    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-885430\""
Sep 14 08:03:07.973212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.973096    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-885430-dockercfg-p4cmh\""
Sep 14 08:03:07.979860 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:07.979837    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"]
Sep 14 08:03:08.051847 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.051810    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-plugins-config-volume\") pod \"payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") " pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.052035 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.051852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtrg8\" (UniqueName: \"kubernetes.io/projected/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-kube-api-access-xtrg8\") pod \"payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") " pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.153212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.153127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-plugins-config-volume\") pod \"payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") " pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.153212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.153170    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtrg8\" (UniqueName: \"kubernetes.io/projected/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-kube-api-access-xtrg8\") pod \"payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") " pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.154125 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.154098    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-plugins-config-volume\") pod \"payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") " pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.160905 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.160875    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtrg8\" (UniqueName: \"kubernetes.io/projected/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-kube-api-access-xtrg8\") pod \"payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") " pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.282571 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.282536    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:08.557335 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.557311    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"]
Sep 14 08:03:08.558909 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:03:08.558884    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6496d3a6_f0d7_4ade_9846_9b7d8575fdda.slice/crio-a38ed509cc7861c9db2c7d02e816c7965f9d8a9a9025332fd6d0de8b6a6b201a WatchSource:0}: Error finding container a38ed509cc7861c9db2c7d02e816c7965f9d8a9a9025332fd6d0de8b6a6b201a: Status 404 returned error can't find the container with id a38ed509cc7861c9db2c7d02e816c7965f9d8a9a9025332fd6d0de8b6a6b201a
Sep 14 08:03:08.861211 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.861124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:08.863497 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.863474    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") pod \"maas-api-e2e-autores-885430-5cb99546df-f6htw\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:08.950169 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.950136    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:08.954519 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:08.954497    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:03:09.015928 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:09.015841    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4" event={"ID":"6496d3a6-f0d7-4ade-9846-9b7d8575fdda","Type":"ContainerStarted","Data":"6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9"}
Sep 14 08:03:09.015928 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:09.015897    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4" event={"ID":"6496d3a6-f0d7-4ade-9846-9b7d8575fdda","Type":"ContainerStarted","Data":"a38ed509cc7861c9db2c7d02e816c7965f9d8a9a9025332fd6d0de8b6a6b201a"}
Sep 14 08:03:09.016827 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:09.016777    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:03:09.037566 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:09.037515    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4" podStartSLOduration=2.037495716 podStartE2EDuration="2.037495716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:09.034450591 +0000 UTC m=+845.533843395" watchObservedRunningTime="2026-09-14 08:03:09.037495716 +0000 UTC m=+845.536888494"
Sep 14 08:03:09.238390 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:09.238362    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"]
Sep 14 08:03:09.239817 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:03:09.239791    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d21f285_f37f_4c3e_a9f3_9a3341381bf6.slice/crio-51baf62b0df40b4447dae16b9b056f560657d83fd58c62eb456e65beb701fe20 WatchSource:0}: Error finding container 51baf62b0df40b4447dae16b9b056f560657d83fd58c62eb456e65beb701fe20: Status 404 returned error can't find the container with id 51baf62b0df40b4447dae16b9b056f560657d83fd58c62eb456e65beb701fe20
Sep 14 08:03:10.026552 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:10.026519    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw" event={"ID":"2d21f285-f37f-4c3e-a9f3-9a3341381bf6","Type":"ContainerStarted","Data":"618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd"}
Sep 14 08:03:10.027008 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:10.026568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw" event={"ID":"2d21f285-f37f-4c3e-a9f3-9a3341381bf6","Type":"ContainerStarted","Data":"51baf62b0df40b4447dae16b9b056f560657d83fd58c62eb456e65beb701fe20"}
Sep 14 08:03:10.027008 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:10.026829    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:10.042037 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:10.041989    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw" podStartSLOduration=2.370397083 podStartE2EDuration="3.041949435s" totalImagesPullingTime="671.552352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:10.041528202 +0000 UTC m=+846.540921000" watchObservedRunningTime="2026-09-14 08:03:10.041949435 +0000 UTC m=+846.541342212"
Sep 14 08:03:16.038407 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:16.038377    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:03:20.029140 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:03:20.029109    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:04:09.263425 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.263391    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"]
Sep 14 08:04:09.267085 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.267055    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.269224 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.269199    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"openshift-service-ca.crt\""
Sep 14 08:04:09.269809 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.269790    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"kube-root-ca.crt\""
Sep 14 08:04:09.269891 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.269848    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"default-dockercfg-4p5xb\""
Sep 14 08:04:09.269891 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.269858    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"simulator-w4-cd7582-kserve-self-signed-certs\""
Sep 14 08:04:09.274190 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.274163    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"]
Sep 14 08:04:09.422849 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.422815    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-tmp-dir\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.423069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.422879    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfqdr\" (UniqueName: \"kubernetes.io/projected/3d34f517-a2b6-4a16-a806-b56716b940f0-kube-api-access-nfqdr\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.423069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.422934    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-home\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.423069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.422987    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d34f517-a2b6-4a16-a806-b56716b940f0-tls-certs\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.423231 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.423055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-model-cache\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.423231 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.423110    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-dshm\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.523886 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.523797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-model-cache\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.523886 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.523858    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-dshm\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524106 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.523902    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-tmp-dir\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524106 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.524006    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfqdr\" (UniqueName: \"kubernetes.io/projected/3d34f517-a2b6-4a16-a806-b56716b940f0-kube-api-access-nfqdr\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524106 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.524040    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-home\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524106 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.524090    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d34f517-a2b6-4a16-a806-b56716b940f0-tls-certs\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524316 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.524240    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-model-cache\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524316 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.524306    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-tmp-dir\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.524431 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.524373    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-home\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.526106 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.526066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-dshm\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.526508 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.526487    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d34f517-a2b6-4a16-a806-b56716b940f0-tls-certs\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.531823 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.531800    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfqdr\" (UniqueName: \"kubernetes.io/projected/3d34f517-a2b6-4a16-a806-b56716b940f0-kube-api-access-nfqdr\") pod \"simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") " pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.578491 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.578461    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:09.851388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:09.851360    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"]
Sep 14 08:04:09.853077 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:04:09.853041    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d34f517_a2b6_4a16_a806_b56716b940f0.slice/crio-254b76b91b4bca93cfc3e6f3f1498ccd4e15aeb47917384b571ac46759e2d1dc WatchSource:0}: Error finding container 254b76b91b4bca93cfc3e6f3f1498ccd4e15aeb47917384b571ac46759e2d1dc: Status 404 returned error can't find the container with id 254b76b91b4bca93cfc3e6f3f1498ccd4e15aeb47917384b571ac46759e2d1dc
Sep 14 08:04:10.356430 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:10.356395    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw" event={"ID":"3d34f517-a2b6-4a16-a806-b56716b940f0","Type":"ContainerStarted","Data":"479b2161f2325c0bd4d25130c33e373a780aaa7a6b7c1a95909dbd6b762149fd"}
Sep 14 08:04:10.356430 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:10.356434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw" event={"ID":"3d34f517-a2b6-4a16-a806-b56716b940f0","Type":"ContainerStarted","Data":"254b76b91b4bca93cfc3e6f3f1498ccd4e15aeb47917384b571ac46759e2d1dc"}
Sep 14 08:04:10.372692 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:10.372640    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw" podStartSLOduration=1.372624238 podStartE2EDuration="1.372624238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:10.37104241 +0000 UTC m=+906.870435215" watchObservedRunningTime="2026-09-14 08:04:10.372624238 +0000 UTC m=+906.872017074"
Sep 14 08:04:19.579027 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:19.578978    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:19.579510 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:19.579044    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:19.591839 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:19.591814    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:04:20.424911 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:04:20.424875    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:05:54.361435 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.361399    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"]
Sep 14 08:05:54.362046 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.361652    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw" podUID="2d21f285-f37f-4c3e-a9f3-9a3341381bf6" containerName="maas-api" containerID="cri-o://618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd" gracePeriod=30
Sep 14 08:05:54.410786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.410750    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"]
Sep 14 08:05:54.411027 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.411000    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4" podUID="6496d3a6-f0d7-4ade-9846-9b7d8575fdda" containerName="payload-processing" containerID="cri-o://6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9" gracePeriod=30
Sep 14 08:05:54.666027 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.665993    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:05:54.704571 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.704544    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:05:54.773047 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.773014    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-metrics-certs\") pod \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") "
Sep 14 08:05:54.773240 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.773056    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") pod \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") "
Sep 14 08:05:54.773240 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.773166    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw5pq\" (UniqueName: \"kubernetes.io/projected/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-kube-api-access-mw5pq\") pod \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") "
Sep 14 08:05:54.773240 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.773224    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-openshift-service-ca\") pod \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\" (UID: \"2d21f285-f37f-4c3e-a9f3-9a3341381bf6\") "
Sep 14 08:05:54.773666 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.773624    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d21f285-f37f-4c3e-a9f3-9a3341381bf6" (UID: "2d21f285-f37f-4c3e-a9f3-9a3341381bf6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:54.775346 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.775314    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2d21f285-f37f-4c3e-a9f3-9a3341381bf6" (UID: "2d21f285-f37f-4c3e-a9f3-9a3341381bf6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:05:54.775648 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.775625    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d21f285-f37f-4c3e-a9f3-9a3341381bf6" (UID: "2d21f285-f37f-4c3e-a9f3-9a3341381bf6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:05:54.775714 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.775638    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-kube-api-access-mw5pq" (OuterVolumeSpecName: "kube-api-access-mw5pq") pod "2d21f285-f37f-4c3e-a9f3-9a3341381bf6" (UID: "2d21f285-f37f-4c3e-a9f3-9a3341381bf6"). InnerVolumeSpecName "kube-api-access-mw5pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:54.874745 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.874652    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-plugins-config-volume\") pod \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") "
Sep 14 08:05:54.874915 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.874782    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtrg8\" (UniqueName: \"kubernetes.io/projected/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-kube-api-access-xtrg8\") pod \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\" (UID: \"6496d3a6-f0d7-4ade-9846-9b7d8575fdda\") "
Sep 14 08:05:54.875053 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.875036    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw5pq\" (UniqueName: \"kubernetes.io/projected/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-kube-api-access-mw5pq\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.875133 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.875058    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.875133 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.875083    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.875133 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.875096    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d21f285-f37f-4c3e-a9f3-9a3341381bf6-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.875242 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.875139    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6496d3a6-f0d7-4ade-9846-9b7d8575fdda" (UID: "6496d3a6-f0d7-4ade-9846-9b7d8575fdda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:54.877006 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.876948    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-kube-api-access-xtrg8" (OuterVolumeSpecName: "kube-api-access-xtrg8") pod "6496d3a6-f0d7-4ade-9846-9b7d8575fdda" (UID: "6496d3a6-f0d7-4ade-9846-9b7d8575fdda"). InnerVolumeSpecName "kube-api-access-xtrg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:54.937278 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.937243    2556 generic.go:356] "Generic (PLEG): container finished" podID="2d21f285-f37f-4c3e-a9f3-9a3341381bf6" containerID="618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd" exitCode=0
Sep 14 08:05:54.937438 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.937285    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"
Sep 14 08:05:54.937438 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.937326    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw" event={"ID":"2d21f285-f37f-4c3e-a9f3-9a3341381bf6","Type":"ContainerDied","Data":"618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd"}
Sep 14 08:05:54.937438 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.937374    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw" event={"ID":"2d21f285-f37f-4c3e-a9f3-9a3341381bf6","Type":"ContainerDied","Data":"51baf62b0df40b4447dae16b9b056f560657d83fd58c62eb456e65beb701fe20"}
Sep 14 08:05:54.937438 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.937396    2556 scope.go:117] "RemoveContainer" containerID="618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd"
Sep 14 08:05:54.938754 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.938731    2556 generic.go:356] "Generic (PLEG): container finished" podID="6496d3a6-f0d7-4ade-9846-9b7d8575fdda" containerID="6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9" exitCode=0
Sep 14 08:05:54.938853 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.938761    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4" event={"ID":"6496d3a6-f0d7-4ade-9846-9b7d8575fdda","Type":"ContainerDied","Data":"6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9"}
Sep 14 08:05:54.938853 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.938809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4" event={"ID":"6496d3a6-f0d7-4ade-9846-9b7d8575fdda","Type":"ContainerDied","Data":"a38ed509cc7861c9db2c7d02e816c7965f9d8a9a9025332fd6d0de8b6a6b201a"}
Sep 14 08:05:54.938853 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.938767    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"
Sep 14 08:05:54.950617 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.950595    2556 scope.go:117] "RemoveContainer" containerID="618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd"
Sep 14 08:05:54.950922 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:05:54.950902    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd\": container with ID starting with 618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd not found: ID does not exist" containerID="618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd"
Sep 14 08:05:54.951020 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.950931    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd"} err="failed to get container status \"618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd\": rpc error: code = NotFound desc = could not find container \"618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd\": container with ID starting with 618de6a5219d7889846c638d84843245986647a72883f763b449307e1e3efdbd not found: ID does not exist"
Sep 14 08:05:54.951020 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.950946    2556 scope.go:117] "RemoveContainer" containerID="6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9"
Sep 14 08:05:54.960660 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.960636    2556 scope.go:117] "RemoveContainer" containerID="6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9"
Sep 14 08:05:54.961435 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:05:54.961045    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9\": container with ID starting with 6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9 not found: ID does not exist" containerID="6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9"
Sep 14 08:05:54.961435 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.961081    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9"} err="failed to get container status \"6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9\": rpc error: code = NotFound desc = could not find container \"6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9\": container with ID starting with 6c7d1a68b31b2fa58e281c5bf41db322d9c9266280d8376a052b216cba2de8d9 not found: ID does not exist"
Sep 14 08:05:54.964003 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.963955    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"]
Sep 14 08:05:54.967389 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.967369    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-885430-5cb99546df-f6htw"]
Sep 14 08:05:54.975521 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.975500    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtrg8\" (UniqueName: \"kubernetes.io/projected/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-kube-api-access-xtrg8\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.975603 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.975524    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6496d3a6-f0d7-4ade-9846-9b7d8575fdda-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.985693 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.985303    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"]
Sep 14 08:05:54.987358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:54.987335    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-885430-5c89b6fdd5-sxdg4"]
Sep 14 08:05:56.019794 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:56.019759    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d21f285-f37f-4c3e-a9f3-9a3341381bf6" path="/var/lib/kubelet/pods/2d21f285-f37f-4c3e-a9f3-9a3341381bf6/volumes"
Sep 14 08:05:56.020177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:05:56.020151    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6496d3a6-f0d7-4ade-9846-9b7d8575fdda" path="/var/lib/kubelet/pods/6496d3a6-f0d7-4ade-9846-9b7d8575fdda/volumes"
Sep 14 08:06:10.183257 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183224    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"]
Sep 14 08:06:10.183652 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183638    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d21f285-f37f-4c3e-a9f3-9a3341381bf6" containerName="maas-api"
Sep 14 08:06:10.183652 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183653    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d21f285-f37f-4c3e-a9f3-9a3341381bf6" containerName="maas-api"
Sep 14 08:06:10.183725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183672    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6496d3a6-f0d7-4ade-9846-9b7d8575fdda" containerName="payload-processing"
Sep 14 08:06:10.183725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183678    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6496d3a6-f0d7-4ade-9846-9b7d8575fdda" containerName="payload-processing"
Sep 14 08:06:10.183788 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183732    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d21f285-f37f-4c3e-a9f3-9a3341381bf6" containerName="maas-api"
Sep 14 08:06:10.183788 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.183744    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6496d3a6-f0d7-4ade-9846-9b7d8575fdda" containerName="payload-processing"
Sep 14 08:06:10.188026 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.188010    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.191092 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.191053    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"openshift-service-ca.crt\""
Sep 14 08:06:10.191092 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.191055    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"default-dockercfg-svws5\""
Sep 14 08:06:10.191298 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.191107    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"premium-w0-831f8e-kserve-self-signed-certs\""
Sep 14 08:06:10.191298 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.191196    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"kube-root-ca.crt\""
Sep 14 08:06:10.196716 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.196691    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"]
Sep 14 08:06:10.208030 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.208002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9l2q\" (UniqueName: \"kubernetes.io/projected/14ce6879-bdf6-4b39-a734-ba5741eed7aa-kube-api-access-n9l2q\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.208166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.208051    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-dshm\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.208166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.208114    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-model-cache\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.208166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.208140    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tls-certs\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.208301 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.208194    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-home\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.208301 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.208225    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tmp-dir\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.308901 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.308858    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tmp-dir\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309097 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.308929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l2q\" (UniqueName: \"kubernetes.io/projected/14ce6879-bdf6-4b39-a734-ba5741eed7aa-kube-api-access-n9l2q\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309097 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.308998    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-dshm\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309097 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.309039    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-model-cache\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309097 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.309060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tls-certs\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309331 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.309110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-home\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309331 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.309313    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tmp-dir\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309455 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.309440    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-model-cache\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.309519 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.309502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-home\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.311393 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.311370    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-dshm\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.311600 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.311585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tls-certs\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.316185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.316163    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9l2q\" (UniqueName: \"kubernetes.io/projected/14ce6879-bdf6-4b39-a734-ba5741eed7aa-kube-api-access-n9l2q\") pod \"premium-w0-831f8e-kserve-578575d967-7nnr8\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") " pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.499169 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.499072    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:10.770284 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:10.770253    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"]
Sep 14 08:06:10.771724 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:06:10.771701    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14ce6879_bdf6_4b39_a734_ba5741eed7aa.slice/crio-1f8b524e05837f07a96b041179ef87d07c574aa17bbd86dcf7b23258d34a784a WatchSource:0}: Error finding container 1f8b524e05837f07a96b041179ef87d07c574aa17bbd86dcf7b23258d34a784a: Status 404 returned error can't find the container with id 1f8b524e05837f07a96b041179ef87d07c574aa17bbd86dcf7b23258d34a784a
Sep 14 08:06:11.032926 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:11.032830    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8" event={"ID":"14ce6879-bdf6-4b39-a734-ba5741eed7aa","Type":"ContainerStarted","Data":"14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f"}
Sep 14 08:06:11.032926 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:11.032880    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8" event={"ID":"14ce6879-bdf6-4b39-a734-ba5741eed7aa","Type":"ContainerStarted","Data":"1f8b524e05837f07a96b041179ef87d07c574aa17bbd86dcf7b23258d34a784a"}
Sep 14 08:06:11.047257 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:11.047203    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8" podStartSLOduration=1.047166663 podStartE2EDuration="1.047166663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:11.046435666 +0000 UTC m=+1027.545828477" watchObservedRunningTime="2026-09-14 08:06:11.047166663 +0000 UTC m=+1027.546559476"
Sep 14 08:06:20.499985 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:20.499927    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:20.500372 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:20.500006    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:20.512537 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:20.512513    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:21.098237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:21.098199    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:06:29.786007 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.785973    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"]
Sep 14 08:06:29.789789 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.789769    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.792226 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.792204    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f67e60\"/\"kube-root-ca.crt\""
Sep 14 08:06:29.792760 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.792745    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f67e60\"/\"openshift-service-ca.crt\""
Sep 14 08:06:29.792861 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.792848    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f67e60\"/\"test-model-f67e60-kserve-self-signed-certs\""
Sep 14 08:06:29.792934 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.792899    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f67e60\"/\"default-dockercfg-npvm4\""
Sep 14 08:06:29.799132 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.799109    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"]
Sep 14 08:06:29.892010 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.891956    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km25z\" (UniqueName: \"kubernetes.io/projected/d0db5baa-2b3e-4920-b72d-6a17a70843ba-kube-api-access-km25z\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.892175 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.892023    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tmp-dir\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.892175 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.892056    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-dshm\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.892175 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.892099    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-model-cache\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.892175 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.892155    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tls-certs\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.892317 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.892195    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-home\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993082 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993050    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-model-cache\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993272 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993106    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tls-certs\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993272 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993146    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-home\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993272 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993171    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km25z\" (UniqueName: \"kubernetes.io/projected/d0db5baa-2b3e-4920-b72d-6a17a70843ba-kube-api-access-km25z\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993272 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tmp-dir\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993483 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993328    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-dshm\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993545 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993509    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-model-cache\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993545 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993526    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-home\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.993648 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.993604    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tmp-dir\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.995495 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.995464    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-dshm\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:29.995696 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:29.995677    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tls-certs\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:30.000700 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:30.000682    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km25z\" (UniqueName: \"kubernetes.io/projected/d0db5baa-2b3e-4920-b72d-6a17a70843ba-kube-api-access-km25z\") pod \"test-model-f67e60-kserve-6855d4dc77-smljd\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") " pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:30.100774 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:30.100687    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:30.374598 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:30.374568    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"]
Sep 14 08:06:30.375695 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:06:30.375665    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0db5baa_2b3e_4920_b72d_6a17a70843ba.slice/crio-87c47cd31543b0b957ff49b94c9c716047b8e0c485cd98f4ef78f25226d52ed2 WatchSource:0}: Error finding container 87c47cd31543b0b957ff49b94c9c716047b8e0c485cd98f4ef78f25226d52ed2: Status 404 returned error can't find the container with id 87c47cd31543b0b957ff49b94c9c716047b8e0c485cd98f4ef78f25226d52ed2
Sep 14 08:06:31.141670 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:31.141629    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd" event={"ID":"d0db5baa-2b3e-4920-b72d-6a17a70843ba","Type":"ContainerStarted","Data":"f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf"}
Sep 14 08:06:31.142176 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:31.141685    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd" event={"ID":"d0db5baa-2b3e-4920-b72d-6a17a70843ba","Type":"ContainerStarted","Data":"87c47cd31543b0b957ff49b94c9c716047b8e0c485cd98f4ef78f25226d52ed2"}
Sep 14 08:06:31.161457 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:31.161403    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd" podStartSLOduration=2.161387546 podStartE2EDuration="2.161387546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:31.159268292 +0000 UTC m=+1047.658661070" watchObservedRunningTime="2026-09-14 08:06:31.161387546 +0000 UTC m=+1047.660780324"
Sep 14 08:06:40.101835 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:40.101793    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:40.101835 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:40.101842    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:40.114665 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:40.114640    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:06:40.203140 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:06:40.203114    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:07:23.967049 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.967010    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"]
Sep 14 08:07:23.971463 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.971439    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:23.974601 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.974578    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-239599\"/\"kube-root-ca.crt\""
Sep 14 08:07:23.974743 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.974631    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-239599\"/\"explicit-model-239599-kserve-self-signed-certs\""
Sep 14 08:07:23.974743 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.974578    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-239599\"/\"default-dockercfg-md9gv\""
Sep 14 08:07:23.974743 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.974589    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-239599\"/\"openshift-service-ca.crt\""
Sep 14 08:07:23.979546 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:23.979521    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"]
Sep 14 08:07:24.075392 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.075357    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-home\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.075545 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.075460    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a451234e-e056-4fa1-8d61-3f06fa9174cf-tls-certs\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.075545 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.075511    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-dshm\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.075545 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.075533    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8w49\" (UniqueName: \"kubernetes.io/projected/a451234e-e056-4fa1-8d61-3f06fa9174cf-kube-api-access-c8w49\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.075656 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.075586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-model-cache\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.075656 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.075611    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-tmp-dir\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176368 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176333    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-model-cache\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176368 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176372    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-tmp-dir\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176590 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176423    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-home\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176590 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176459    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a451234e-e056-4fa1-8d61-3f06fa9174cf-tls-certs\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-dshm\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8w49\" (UniqueName: \"kubernetes.io/projected/a451234e-e056-4fa1-8d61-3f06fa9174cf-kube-api-access-c8w49\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176769    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-model-cache\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176787    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-tmp-dir\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.176872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.176855    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-home\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.178738 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.178711    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-dshm\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.179145 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.179121    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a451234e-e056-4fa1-8d61-3f06fa9174cf-tls-certs\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.189665 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.189637    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8w49\" (UniqueName: \"kubernetes.io/projected/a451234e-e056-4fa1-8d61-3f06fa9174cf-kube-api-access-c8w49\") pod \"explicit-model-239599-kserve-647b69694c-lrj6q\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") " pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.282393 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.282320    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:24.570369 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:24.570327    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"]
Sep 14 08:07:24.571654 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:07:24.571629    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda451234e_e056_4fa1_8d61_3f06fa9174cf.slice/crio-b585064e395c12e1f95306c44ec4a53cdb1f5a87a3ff1666d6dc0d9cd5d903d5 WatchSource:0}: Error finding container b585064e395c12e1f95306c44ec4a53cdb1f5a87a3ff1666d6dc0d9cd5d903d5: Status 404 returned error can't find the container with id b585064e395c12e1f95306c44ec4a53cdb1f5a87a3ff1666d6dc0d9cd5d903d5
Sep 14 08:07:25.446502 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:25.446466    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q" event={"ID":"a451234e-e056-4fa1-8d61-3f06fa9174cf","Type":"ContainerStarted","Data":"bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62"}
Sep 14 08:07:25.446877 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:25.446511    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q" event={"ID":"a451234e-e056-4fa1-8d61-3f06fa9174cf","Type":"ContainerStarted","Data":"b585064e395c12e1f95306c44ec4a53cdb1f5a87a3ff1666d6dc0d9cd5d903d5"}
Sep 14 08:07:25.462411 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:25.462368    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q" podStartSLOduration=2.462354741 podStartE2EDuration="2.462354741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:25.461615997 +0000 UTC m=+1101.961008792" watchObservedRunningTime="2026-09-14 08:07:25.462354741 +0000 UTC m=+1101.961747518"
Sep 14 08:07:28.721369 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.721323    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"]
Sep 14 08:07:28.724941 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.724924    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:28.726925 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.726905    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-58a5b128-dockercfg-xg7bh\""
Sep 14 08:07:28.727046 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.726908    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-58a5b128\""
Sep 14 08:07:28.734846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.734822    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"]
Sep 14 08:07:28.822857 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.822820    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f869da-e555-4164-8715-32a48c82c44f-plugins-config-volume\") pod \"payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") " pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:28.823027 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.822979    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pcnk\" (UniqueName: \"kubernetes.io/projected/b4f869da-e555-4164-8715-32a48c82c44f-kube-api-access-2pcnk\") pod \"payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") " pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:28.924161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.924122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f869da-e555-4164-8715-32a48c82c44f-plugins-config-volume\") pod \"payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") " pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:28.924314 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.924229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcnk\" (UniqueName: \"kubernetes.io/projected/b4f869da-e555-4164-8715-32a48c82c44f-kube-api-access-2pcnk\") pod \"payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") " pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:28.924810 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.924790    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f869da-e555-4164-8715-32a48c82c44f-plugins-config-volume\") pod \"payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") " pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:28.931357 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:28.931332    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pcnk\" (UniqueName: \"kubernetes.io/projected/b4f869da-e555-4164-8715-32a48c82c44f-kube-api-access-2pcnk\") pod \"payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") " pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:29.037517 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:29.037440    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:29.309077 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:29.309049    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"]
Sep 14 08:07:29.310593 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:07:29.310567    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f869da_e555_4164_8715_32a48c82c44f.slice/crio-b23ab07e7ada2b3ffae94554769101df6387b703f730956ae3da7bcf41a925e7 WatchSource:0}: Error finding container b23ab07e7ada2b3ffae94554769101df6387b703f730956ae3da7bcf41a925e7: Status 404 returned error can't find the container with id b23ab07e7ada2b3ffae94554769101df6387b703f730956ae3da7bcf41a925e7
Sep 14 08:07:29.471002 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:29.470941    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f" event={"ID":"b4f869da-e555-4164-8715-32a48c82c44f","Type":"ContainerStarted","Data":"3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac"}
Sep 14 08:07:29.471195 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:29.471029    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f" event={"ID":"b4f869da-e555-4164-8715-32a48c82c44f","Type":"ContainerStarted","Data":"b23ab07e7ada2b3ffae94554769101df6387b703f730956ae3da7bcf41a925e7"}
Sep 14 08:07:29.471195 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:29.471178    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:29.486682 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:29.486632    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f" podStartSLOduration=1.486619076 podStartE2EDuration="1.486619076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:29.484894155 +0000 UTC m=+1105.984286930" watchObservedRunningTime="2026-09-14 08:07:29.486619076 +0000 UTC m=+1105.986011853"
Sep 14 08:07:32.860374 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:32.860341    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"]
Sep 14 08:07:34.283093 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:34.283050    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:40.478621 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:40.478589    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:47.879526 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:47.879481    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q" podUID="a451234e-e056-4fa1-8d61-3f06fa9174cf" containerName="main" containerID="cri-o://bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62" gracePeriod=45
Sep 14 08:07:48.141296 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.141240    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:48.198559 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198527    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-model-cache\") pod \"a451234e-e056-4fa1-8d61-3f06fa9174cf\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") "
Sep 14 08:07:48.198712 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198653    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-dshm\") pod \"a451234e-e056-4fa1-8d61-3f06fa9174cf\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") "
Sep 14 08:07:48.198712 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198689    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a451234e-e056-4fa1-8d61-3f06fa9174cf-tls-certs\") pod \"a451234e-e056-4fa1-8d61-3f06fa9174cf\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") "
Sep 14 08:07:48.198712 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198698    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "a451234e-e056-4fa1-8d61-3f06fa9174cf" (UID: "a451234e-e056-4fa1-8d61-3f06fa9174cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:48.198903 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198746    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8w49\" (UniqueName: \"kubernetes.io/projected/a451234e-e056-4fa1-8d61-3f06fa9174cf-kube-api-access-c8w49\") pod \"a451234e-e056-4fa1-8d61-3f06fa9174cf\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") "
Sep 14 08:07:48.198903 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198791    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-home\") pod \"a451234e-e056-4fa1-8d61-3f06fa9174cf\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") "
Sep 14 08:07:48.198903 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.198821    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-tmp-dir\") pod \"a451234e-e056-4fa1-8d61-3f06fa9174cf\" (UID: \"a451234e-e056-4fa1-8d61-3f06fa9174cf\") "
Sep 14 08:07:48.199177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.199159    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:48.199252 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.199171    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a451234e-e056-4fa1-8d61-3f06fa9174cf" (UID: "a451234e-e056-4fa1-8d61-3f06fa9174cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:48.199252 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.199179    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-home" (OuterVolumeSpecName: "home") pod "a451234e-e056-4fa1-8d61-3f06fa9174cf" (UID: "a451234e-e056-4fa1-8d61-3f06fa9174cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:48.201034 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.201010    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a451234e-e056-4fa1-8d61-3f06fa9174cf-kube-api-access-c8w49" (OuterVolumeSpecName: "kube-api-access-c8w49") pod "a451234e-e056-4fa1-8d61-3f06fa9174cf" (UID: "a451234e-e056-4fa1-8d61-3f06fa9174cf"). InnerVolumeSpecName "kube-api-access-c8w49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:48.201240 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.201223    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-dshm" (OuterVolumeSpecName: "dshm") pod "a451234e-e056-4fa1-8d61-3f06fa9174cf" (UID: "a451234e-e056-4fa1-8d61-3f06fa9174cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:48.201294 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.201243    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a451234e-e056-4fa1-8d61-3f06fa9174cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a451234e-e056-4fa1-8d61-3f06fa9174cf" (UID: "a451234e-e056-4fa1-8d61-3f06fa9174cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:48.300246 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.300209    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:48.300246 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.300238    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a451234e-e056-4fa1-8d61-3f06fa9174cf-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:48.300246 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.300251    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8w49\" (UniqueName: \"kubernetes.io/projected/a451234e-e056-4fa1-8d61-3f06fa9174cf-kube-api-access-c8w49\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:48.300477 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.300261    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:48.300477 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.300271    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a451234e-e056-4fa1-8d61-3f06fa9174cf-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:48.578358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.578315    2556 generic.go:356] "Generic (PLEG): container finished" podID="a451234e-e056-4fa1-8d61-3f06fa9174cf" containerID="bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62" exitCode=0
Sep 14 08:07:48.578586 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.578360    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"
Sep 14 08:07:48.578586 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.578382    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q" event={"ID":"a451234e-e056-4fa1-8d61-3f06fa9174cf","Type":"ContainerDied","Data":"bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62"}
Sep 14 08:07:48.578586 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.578431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q" event={"ID":"a451234e-e056-4fa1-8d61-3f06fa9174cf","Type":"ContainerDied","Data":"b585064e395c12e1f95306c44ec4a53cdb1f5a87a3ff1666d6dc0d9cd5d903d5"}
Sep 14 08:07:48.578586 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.578450    2556 scope.go:117] "RemoveContainer" containerID="bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62"
Sep 14 08:07:48.588391 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.588373    2556 scope.go:117] "RemoveContainer" containerID="bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62"
Sep 14 08:07:48.588674 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:07:48.588651    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62\": container with ID starting with bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62 not found: ID does not exist" containerID="bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62"
Sep 14 08:07:48.588733 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.588689    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62"} err="failed to get container status \"bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62\": rpc error: code = NotFound desc = could not find container \"bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62\": container with ID starting with bd81c03fe57c2aa035d3025306cd597715dc5ddad80695b30d23b88eea60ef62 not found: ID does not exist"
Sep 14 08:07:48.600255 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.600226    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"]
Sep 14 08:07:48.609620 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:48.609590    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-239599/explicit-model-239599-kserve-647b69694c-lrj6q"]
Sep 14 08:07:50.020568 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:50.020523    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a451234e-e056-4fa1-8d61-3f06fa9174cf" path="/var/lib/kubelet/pods/a451234e-e056-4fa1-8d61-3f06fa9174cf/volumes"
Sep 14 08:07:56.094773 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.094740    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"]
Sep 14 08:07:56.095662 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.095609    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f" podUID="b4f869da-e555-4164-8715-32a48c82c44f" containerName="payload-processing" containerID="cri-o://3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac" gracePeriod=30
Sep 14 08:07:56.366945 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.366910    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:56.470353 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.470321    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f869da-e555-4164-8715-32a48c82c44f-plugins-config-volume\") pod \"b4f869da-e555-4164-8715-32a48c82c44f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") "
Sep 14 08:07:56.470544 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.470365    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pcnk\" (UniqueName: \"kubernetes.io/projected/b4f869da-e555-4164-8715-32a48c82c44f-kube-api-access-2pcnk\") pod \"b4f869da-e555-4164-8715-32a48c82c44f\" (UID: \"b4f869da-e555-4164-8715-32a48c82c44f\") "
Sep 14 08:07:56.470766 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.470737    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4f869da-e555-4164-8715-32a48c82c44f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4f869da-e555-4164-8715-32a48c82c44f" (UID: "b4f869da-e555-4164-8715-32a48c82c44f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:56.472492 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.472467    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4f869da-e555-4164-8715-32a48c82c44f-kube-api-access-2pcnk" (OuterVolumeSpecName: "kube-api-access-2pcnk") pod "b4f869da-e555-4164-8715-32a48c82c44f" (UID: "b4f869da-e555-4164-8715-32a48c82c44f"). InnerVolumeSpecName "kube-api-access-2pcnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:56.571428 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.571397    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4f869da-e555-4164-8715-32a48c82c44f-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.571428 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.571426    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pcnk\" (UniqueName: \"kubernetes.io/projected/b4f869da-e555-4164-8715-32a48c82c44f-kube-api-access-2pcnk\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.629109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.629020    2556 generic.go:356] "Generic (PLEG): container finished" podID="b4f869da-e555-4164-8715-32a48c82c44f" containerID="3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac" exitCode=0
Sep 14 08:07:56.629109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.629069    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"
Sep 14 08:07:56.629292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.629112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f" event={"ID":"b4f869da-e555-4164-8715-32a48c82c44f","Type":"ContainerDied","Data":"3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac"}
Sep 14 08:07:56.629292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.629165    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f" event={"ID":"b4f869da-e555-4164-8715-32a48c82c44f","Type":"ContainerDied","Data":"b23ab07e7ada2b3ffae94554769101df6387b703f730956ae3da7bcf41a925e7"}
Sep 14 08:07:56.629292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.629186    2556 scope.go:117] "RemoveContainer" containerID="3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac"
Sep 14 08:07:56.638462 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.638442    2556 scope.go:117] "RemoveContainer" containerID="3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac"
Sep 14 08:07:56.638748 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:07:56.638728    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac\": container with ID starting with 3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac not found: ID does not exist" containerID="3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac"
Sep 14 08:07:56.638823 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.638758    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac"} err="failed to get container status \"3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac\": rpc error: code = NotFound desc = could not find container \"3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac\": container with ID starting with 3fc629395a29bac92e246db653159e92be919eecbf94dc48310d2dbe7dc37bac not found: ID does not exist"
Sep 14 08:07:56.649950 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.649928    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"]
Sep 14 08:07:56.654558 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:56.654536    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58a5b128-67b46984b5-lhq4f"]
Sep 14 08:07:58.020126 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:07:58.020092    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4f869da-e555-4164-8715-32a48c82c44f" path="/var/lib/kubelet/pods/b4f869da-e555-4164-8715-32a48c82c44f/volumes"
Sep 14 08:08:39.582830 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.582747    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"]
Sep 14 08:08:39.583427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.583275    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f869da-e555-4164-8715-32a48c82c44f" containerName="payload-processing"
Sep 14 08:08:39.583427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.583292    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f869da-e555-4164-8715-32a48c82c44f" containerName="payload-processing"
Sep 14 08:08:39.583427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.583308    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a451234e-e056-4fa1-8d61-3f06fa9174cf" containerName="main"
Sep 14 08:08:39.583427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.583314    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a451234e-e056-4fa1-8d61-3f06fa9174cf" containerName="main"
Sep 14 08:08:39.583427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.583377    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a451234e-e056-4fa1-8d61-3f06fa9174cf" containerName="main"
Sep 14 08:08:39.583427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.583386    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f869da-e555-4164-8715-32a48c82c44f" containerName="payload-processing"
Sep 14 08:08:39.586448 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.586426    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.588689 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.588666    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"distinct-w0-831f8e-kserve-self-signed-certs\""
Sep 14 08:08:39.595368 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.595344    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"]
Sep 14 08:08:39.672448 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.672411    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-model-cache\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.672448 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.672460    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-tmp-dir\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.672699 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.672537    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7347bf8e-20ee-418f-9eb3-afec682d8517-tls-certs\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.672699 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.672575    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-dshm\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.672699 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.672627    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjb6l\" (UniqueName: \"kubernetes.io/projected/7347bf8e-20ee-418f-9eb3-afec682d8517-kube-api-access-cjb6l\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.672699 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.672685    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-home\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.773627 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.773594    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7347bf8e-20ee-418f-9eb3-afec682d8517-tls-certs\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.773790 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.773635    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-dshm\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.773790 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.773677    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjb6l\" (UniqueName: \"kubernetes.io/projected/7347bf8e-20ee-418f-9eb3-afec682d8517-kube-api-access-cjb6l\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.773790 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.773713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-home\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.773790 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.773738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-model-cache\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.773790 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.773766    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-tmp-dir\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.774147 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.774123    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-model-cache\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.774225 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.774206    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-tmp-dir\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.774284 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.774259    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-home\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.775866 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.775848    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-dshm\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.776227 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.776211    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7347bf8e-20ee-418f-9eb3-afec682d8517-tls-certs\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.780642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.780623    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjb6l\" (UniqueName: \"kubernetes.io/projected/7347bf8e-20ee-418f-9eb3-afec682d8517-kube-api-access-cjb6l\") pod \"distinct-w0-831f8e-kserve-788546bbb-pq6ls\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:39.898243 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:39.898157    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:40.171073 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:40.171038    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"]
Sep 14 08:08:40.172913 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:08:40.172887    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7347bf8e_20ee_418f_9eb3_afec682d8517.slice/crio-8c49fdbab2da532ea9712f8963e29b61905ceaf8a3ae7a3a52d3a48c5054db61 WatchSource:0}: Error finding container 8c49fdbab2da532ea9712f8963e29b61905ceaf8a3ae7a3a52d3a48c5054db61: Status 404 returned error can't find the container with id 8c49fdbab2da532ea9712f8963e29b61905ceaf8a3ae7a3a52d3a48c5054db61
Sep 14 08:08:40.875810 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:40.875773    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls" event={"ID":"7347bf8e-20ee-418f-9eb3-afec682d8517","Type":"ContainerStarted","Data":"b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512"}
Sep 14 08:08:40.876207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:40.875817    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls" event={"ID":"7347bf8e-20ee-418f-9eb3-afec682d8517","Type":"ContainerStarted","Data":"8c49fdbab2da532ea9712f8963e29b61905ceaf8a3ae7a3a52d3a48c5054db61"}
Sep 14 08:08:40.893225 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:40.893179    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls" podStartSLOduration=1.893163764 podStartE2EDuration="1.893163764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:40.890365875 +0000 UTC m=+1177.389758655" watchObservedRunningTime="2026-09-14 08:08:40.893163764 +0000 UTC m=+1177.392556541"
Sep 14 08:08:47.474595 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.474555    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"]
Sep 14 08:08:47.484339 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.484313    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"]
Sep 14 08:08:47.484513 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.484426    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.486497 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.486473    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"e2e-conflict-b-2827d38c-kserve-self-signed-certs\""
Sep 14 08:08:47.548281 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.548248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-dshm\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.548281 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.548282    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-model-cache\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.548498 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.548302    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-home\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.548498 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.548322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-tmp-dir\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.548498 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.548377    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq8tj\" (UniqueName: \"kubernetes.io/projected/f56c2a94-430b-46e1-be42-9e112fd09fa4-kube-api-access-mq8tj\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.548498 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.548485    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56c2a94-430b-46e1-be42-9e112fd09fa4-tls-certs\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.649704 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.649666    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-dshm\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.649704 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.649706    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-model-cache\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.649946 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.649724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-home\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.649946 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.649754    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-tmp-dir\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.649946 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.649779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq8tj\" (UniqueName: \"kubernetes.io/projected/f56c2a94-430b-46e1-be42-9e112fd09fa4-kube-api-access-mq8tj\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.649946 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.649825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56c2a94-430b-46e1-be42-9e112fd09fa4-tls-certs\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.650182 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.650151    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-tmp-dir\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.650237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.650155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-home\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.650280 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.650239    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-model-cache\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.652085 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.652058    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-dshm\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.652488 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.652465    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56c2a94-430b-46e1-be42-9e112fd09fa4-tls-certs\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.657167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.657139    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq8tj\" (UniqueName: \"kubernetes.io/projected/f56c2a94-430b-46e1-be42-9e112fd09fa4-kube-api-access-mq8tj\") pod \"e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") " pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:47.795178 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:47.795095    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:48.070896 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:48.070866    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"]
Sep 14 08:08:48.072311 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:08:48.072288    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf56c2a94_430b_46e1_be42_9e112fd09fa4.slice/crio-8b7129e04c93ca387e929aefed3bc75b7c602bc7675b89f6e2718ab2d2637b06 WatchSource:0}: Error finding container 8b7129e04c93ca387e929aefed3bc75b7c602bc7675b89f6e2718ab2d2637b06: Status 404 returned error can't find the container with id 8b7129e04c93ca387e929aefed3bc75b7c602bc7675b89f6e2718ab2d2637b06
Sep 14 08:08:48.920830 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:48.920790    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4" event={"ID":"f56c2a94-430b-46e1-be42-9e112fd09fa4","Type":"ContainerStarted","Data":"81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5"}
Sep 14 08:08:48.920830 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:48.920838    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4" event={"ID":"f56c2a94-430b-46e1-be42-9e112fd09fa4","Type":"ContainerStarted","Data":"8b7129e04c93ca387e929aefed3bc75b7c602bc7675b89f6e2718ab2d2637b06"}
Sep 14 08:08:48.938291 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:48.938239    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4" podStartSLOduration=1.938223889 podStartE2EDuration="1.938223889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:48.935399216 +0000 UTC m=+1185.434792012" watchObservedRunningTime="2026-09-14 08:08:48.938223889 +0000 UTC m=+1185.437616667"
Sep 14 08:08:49.898715 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:49.898680    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:49.898896 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:49.898726    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:49.911620 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:49.911589    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:49.937678 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:49.937651    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:08:54.265779 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:54.265745    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"]
Sep 14 08:08:57.795845 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:57.795805    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:57.796281 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:57.795862    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:57.809021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:57.808993    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:08:57.982325 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:08:57.982290    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:09:05.173151 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.173114    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"]
Sep 14 08:09:05.173726 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.173678    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" podUID="afec4335-6dc4-4f6e-bd54-f1a0a00ff253" containerName="maas-api" containerID="cri-o://5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8" gracePeriod=30
Sep 14 08:09:05.230562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.230522    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"]
Sep 14 08:09:05.230833 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.230775    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz" podUID="18f10aff-5699-4147-aa25-7e9326a1757c" containerName="payload-processing" containerID="cri-o://a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3" gracePeriod=30
Sep 14 08:09:05.237636 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.237602    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"]
Sep 14 08:09:05.237879 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.237845    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c" podUID="f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" containerName="payload-pre-processing" containerID="cri-o://53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf" gracePeriod=30
Sep 14 08:09:05.548185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.548157    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:09:05.550775 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.550753    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:09:05.554060 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.554041    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:09:05.723330 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723240    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kfbw\" (UniqueName: \"kubernetes.io/projected/18f10aff-5699-4147-aa25-7e9326a1757c-kube-api-access-9kfbw\") pod \"18f10aff-5699-4147-aa25-7e9326a1757c\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") "
Sep 14 08:09:05.723330 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723307    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f10aff-5699-4147-aa25-7e9326a1757c-plugins-config-volume\") pod \"18f10aff-5699-4147-aa25-7e9326a1757c\" (UID: \"18f10aff-5699-4147-aa25-7e9326a1757c\") "
Sep 14 08:09:05.723578 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723346    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvvzm\" (UniqueName: \"kubernetes.io/projected/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-kube-api-access-tvvzm\") pod \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") "
Sep 14 08:09:05.723578 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723366    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls\") pod \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") "
Sep 14 08:09:05.723578 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723397    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqz2r\" (UniqueName: \"kubernetes.io/projected/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-kube-api-access-wqz2r\") pod \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") "
Sep 14 08:09:05.723578 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723485    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-openshift-service-ca\") pod \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") "
Sep 14 08:09:05.723578 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723516    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-metrics-certs\") pod \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\" (UID: \"afec4335-6dc4-4f6e-bd54-f1a0a00ff253\") "
Sep 14 08:09:05.723836 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723607    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-plugins-config-volume\") pod \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\" (UID: \"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098\") "
Sep 14 08:09:05.723836 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723691    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f10aff-5699-4147-aa25-7e9326a1757c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18f10aff-5699-4147-aa25-7e9326a1757c" (UID: "18f10aff-5699-4147-aa25-7e9326a1757c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:05.723947 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.723863    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "afec4335-6dc4-4f6e-bd54-f1a0a00ff253" (UID: "afec4335-6dc4-4f6e-bd54-f1a0a00ff253"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:05.724185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.724124    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.724185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.724151    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f10aff-5699-4147-aa25-7e9326a1757c-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.724185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.724166    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" (UID: "f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:05.725652 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.725619    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-kube-api-access-wqz2r" (OuterVolumeSpecName: "kube-api-access-wqz2r") pod "f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" (UID: "f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098"). InnerVolumeSpecName "kube-api-access-wqz2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:05.725652 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.725633    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "afec4335-6dc4-4f6e-bd54-f1a0a00ff253" (UID: "afec4335-6dc4-4f6e-bd54-f1a0a00ff253"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:05.725793 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.725643    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f10aff-5699-4147-aa25-7e9326a1757c-kube-api-access-9kfbw" (OuterVolumeSpecName: "kube-api-access-9kfbw") pod "18f10aff-5699-4147-aa25-7e9326a1757c" (UID: "18f10aff-5699-4147-aa25-7e9326a1757c"). InnerVolumeSpecName "kube-api-access-9kfbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:05.725906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.725888    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "afec4335-6dc4-4f6e-bd54-f1a0a00ff253" (UID: "afec4335-6dc4-4f6e-bd54-f1a0a00ff253"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:05.725944 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.725925    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-kube-api-access-tvvzm" (OuterVolumeSpecName: "kube-api-access-tvvzm") pod "afec4335-6dc4-4f6e-bd54-f1a0a00ff253" (UID: "afec4335-6dc4-4f6e-bd54-f1a0a00ff253"). InnerVolumeSpecName "kube-api-access-tvvzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:05.824781 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.824749    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.824781 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.824776    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kfbw\" (UniqueName: \"kubernetes.io/projected/18f10aff-5699-4147-aa25-7e9326a1757c-kube-api-access-9kfbw\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.824781 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.824786    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvvzm\" (UniqueName: \"kubernetes.io/projected/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-kube-api-access-tvvzm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.825020 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.824795    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.825020 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.824804    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqz2r\" (UniqueName: \"kubernetes.io/projected/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098-kube-api-access-wqz2r\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:05.825020 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:05.824813    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afec4335-6dc4-4f6e-bd54-f1a0a00ff253-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:06.016561 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.016475    2556 generic.go:356] "Generic (PLEG): container finished" podID="18f10aff-5699-4147-aa25-7e9326a1757c" containerID="a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3" exitCode=0
Sep 14 08:09:06.016706 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.016573    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"
Sep 14 08:09:06.018015 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.017987    2556 generic.go:356] "Generic (PLEG): container finished" podID="f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" containerID="53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf" exitCode=0
Sep 14 08:09:06.018142 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.018021    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"
Sep 14 08:09:06.019436 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019409    2556 generic.go:356] "Generic (PLEG): container finished" podID="afec4335-6dc4-4f6e-bd54-f1a0a00ff253" containerID="5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8" exitCode=0
Sep 14 08:09:06.019539 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019509    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019841    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz" event={"ID":"18f10aff-5699-4147-aa25-7e9326a1757c","Type":"ContainerDied","Data":"a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3"}
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019888    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz" event={"ID":"18f10aff-5699-4147-aa25-7e9326a1757c","Type":"ContainerDied","Data":"201f7d53d417223fc0aa7d8ad8ac1e610287ef2060405fac5ebd0dc37d12d17a"}
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019913    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c" event={"ID":"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098","Type":"ContainerDied","Data":"53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf"}
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019936    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c" event={"ID":"f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098","Type":"ContainerDied","Data":"13b032d4ededb8a6dfb11c7aa820eaeb6e09c79ee741762072aad311c38d6d88"}
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019958    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" event={"ID":"afec4335-6dc4-4f6e-bd54-f1a0a00ff253","Type":"ContainerDied","Data":"5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8"}
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.019998    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt" event={"ID":"afec4335-6dc4-4f6e-bd54-f1a0a00ff253","Type":"ContainerDied","Data":"847ab0e33a54d1786ae68368d2d2818d8616d5decdbf0996cb62fc92d09de1ab"}
Sep 14 08:09:06.020237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.020027    2556 scope.go:117] "RemoveContainer" containerID="a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3"
Sep 14 08:09:06.041801 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.041781    2556 scope.go:117] "RemoveContainer" containerID="a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3"
Sep 14 08:09:06.042092 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:06.042069    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3\": container with ID starting with a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3 not found: ID does not exist" containerID="a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3"
Sep 14 08:09:06.042181 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.042100    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3"} err="failed to get container status \"a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3\": rpc error: code = NotFound desc = could not find container \"a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3\": container with ID starting with a5ee32e1111b887602dc63ffe207a0c3a532b96bbb7ee15509c5c60f4b98e6a3 not found: ID does not exist"
Sep 14 08:09:06.042181 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.042117    2556 scope.go:117] "RemoveContainer" containerID="53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf"
Sep 14 08:09:06.047840 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.047820    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"]
Sep 14 08:09:06.050873 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.050855    2556 scope.go:117] "RemoveContainer" containerID="53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf"
Sep 14 08:09:06.051234 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:06.051214    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf\": container with ID starting with 53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf not found: ID does not exist" containerID="53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf"
Sep 14 08:09:06.051318 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.051238    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf"} err="failed to get container status \"53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf\": rpc error: code = NotFound desc = could not find container \"53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf\": container with ID starting with 53ca982219b628a564b4b3a537453811c186442baa913b65f68422800c2cbcaf not found: ID does not exist"
Sep 14 08:09:06.051318 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.051254    2556 scope.go:117] "RemoveContainer" containerID="5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8"
Sep 14 08:09:06.054486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.054466    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e7b0a8-9b588d55d-6d64c"]
Sep 14 08:09:06.060066 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.060045    2556 scope.go:117] "RemoveContainer" containerID="5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8"
Sep 14 08:09:06.060309 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:06.060291    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8\": container with ID starting with 5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8 not found: ID does not exist" containerID="5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8"
Sep 14 08:09:06.060354 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.060320    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8"} err="failed to get container status \"5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8\": rpc error: code = NotFound desc = could not find container \"5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8\": container with ID starting with 5231f2b65f881a5fb53c20b28337734e91979ee27548a792d99379d5ffeb26a8 not found: ID does not exist"
Sep 14 08:09:06.065565 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.065547    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"]
Sep 14 08:09:06.069650 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.069631    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e7b0a8-867495c547-kkwzz"]
Sep 14 08:09:06.079083 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.079063    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"]
Sep 14 08:09:06.082883 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:06.082862    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7b0a8-767dcc595b-n2kjt"]
Sep 14 08:09:08.018982 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:08.018940    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f10aff-5699-4147-aa25-7e9326a1757c" path="/var/lib/kubelet/pods/18f10aff-5699-4147-aa25-7e9326a1757c/volumes"
Sep 14 08:09:08.019410 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:08.019394    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afec4335-6dc4-4f6e-bd54-f1a0a00ff253" path="/var/lib/kubelet/pods/afec4335-6dc4-4f6e-bd54-f1a0a00ff253/volumes"
Sep 14 08:09:08.019716 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:08.019704    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" path="/var/lib/kubelet/pods/f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098/volumes"
Sep 14 08:09:09.287634 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.287553    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd" podUID="d0db5baa-2b3e-4920-b72d-6a17a70843ba" containerName="main" containerID="cri-o://f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf" gracePeriod=45
Sep 14 08:09:09.542947 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.542887    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:09:09.663447 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663398    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-home\") pod \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") "
Sep 14 08:09:09.663447 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663456    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-dshm\") pod \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") "
Sep 14 08:09:09.663725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663480    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km25z\" (UniqueName: \"kubernetes.io/projected/d0db5baa-2b3e-4920-b72d-6a17a70843ba-kube-api-access-km25z\") pod \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") "
Sep 14 08:09:09.663725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663503    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tmp-dir\") pod \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") "
Sep 14 08:09:09.663725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663561    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-model-cache\") pod \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") "
Sep 14 08:09:09.663725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663658    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tls-certs\") pod \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\" (UID: \"d0db5baa-2b3e-4920-b72d-6a17a70843ba\") "
Sep 14 08:09:09.663725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663668    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-home" (OuterVolumeSpecName: "home") pod "d0db5baa-2b3e-4920-b72d-6a17a70843ba" (UID: "d0db5baa-2b3e-4920-b72d-6a17a70843ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:09.664015 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663863    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0db5baa-2b3e-4920-b72d-6a17a70843ba" (UID: "d0db5baa-2b3e-4920-b72d-6a17a70843ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:09.664015 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.663876    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0db5baa-2b3e-4920-b72d-6a17a70843ba" (UID: "d0db5baa-2b3e-4920-b72d-6a17a70843ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:09.664166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.664087    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:09.664166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.664107    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:09.664166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.664121    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:09.665887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.665861    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0db5baa-2b3e-4920-b72d-6a17a70843ba-kube-api-access-km25z" (OuterVolumeSpecName: "kube-api-access-km25z") pod "d0db5baa-2b3e-4920-b72d-6a17a70843ba" (UID: "d0db5baa-2b3e-4920-b72d-6a17a70843ba"). InnerVolumeSpecName "kube-api-access-km25z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:09.665887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.665878    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-dshm" (OuterVolumeSpecName: "dshm") pod "d0db5baa-2b3e-4920-b72d-6a17a70843ba" (UID: "d0db5baa-2b3e-4920-b72d-6a17a70843ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:09.666053 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.665888    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0db5baa-2b3e-4920-b72d-6a17a70843ba" (UID: "d0db5baa-2b3e-4920-b72d-6a17a70843ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:09.764942 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.764897    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0db5baa-2b3e-4920-b72d-6a17a70843ba-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:09.764942 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.764938    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km25z\" (UniqueName: \"kubernetes.io/projected/d0db5baa-2b3e-4920-b72d-6a17a70843ba-kube-api-access-km25z\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:09.765167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:09.764954    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0db5baa-2b3e-4920-b72d-6a17a70843ba-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:10.051467 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.051422    2556 generic.go:356] "Generic (PLEG): container finished" podID="d0db5baa-2b3e-4920-b72d-6a17a70843ba" containerID="f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf" exitCode=0
Sep 14 08:09:10.051467 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.051462    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"
Sep 14 08:09:10.051668 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.051514    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd" event={"ID":"d0db5baa-2b3e-4920-b72d-6a17a70843ba","Type":"ContainerDied","Data":"f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf"}
Sep 14 08:09:10.051668 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.051551    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd" event={"ID":"d0db5baa-2b3e-4920-b72d-6a17a70843ba","Type":"ContainerDied","Data":"87c47cd31543b0b957ff49b94c9c716047b8e0c485cd98f4ef78f25226d52ed2"}
Sep 14 08:09:10.051668 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.051569    2556 scope.go:117] "RemoveContainer" containerID="f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf"
Sep 14 08:09:10.060831 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.060812    2556 scope.go:117] "RemoveContainer" containerID="f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf"
Sep 14 08:09:10.061141 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:10.061121    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf\": container with ID starting with f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf not found: ID does not exist" containerID="f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf"
Sep 14 08:09:10.061199 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.061149    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf"} err="failed to get container status \"f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf\": rpc error: code = NotFound desc = could not find container \"f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf\": container with ID starting with f0ca34e15db5766c790fa00669d01e6367e355b7e7e0f02038d26a7f98dedadf not found: ID does not exist"
Sep 14 08:09:10.067846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.067823    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"]
Sep 14 08:09:10.070181 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:10.070161    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f67e60/test-model-f67e60-kserve-6855d4dc77-smljd"]
Sep 14 08:09:12.019077 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:12.019032    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0db5baa-2b3e-4920-b72d-6a17a70843ba" path="/var/lib/kubelet/pods/d0db5baa-2b3e-4920-b72d-6a17a70843ba/volumes"
Sep 14 08:09:14.198576 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198546    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-7bda51"]
Sep 14 08:09:14.198935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198901    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f10aff-5699-4147-aa25-7e9326a1757c" containerName="payload-processing"
Sep 14 08:09:14.198935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198912    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f10aff-5699-4147-aa25-7e9326a1757c" containerName="payload-processing"
Sep 14 08:09:14.198935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198924    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afec4335-6dc4-4f6e-bd54-f1a0a00ff253" containerName="maas-api"
Sep 14 08:09:14.198935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198930    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="afec4335-6dc4-4f6e-bd54-f1a0a00ff253" containerName="maas-api"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198948    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" containerName="payload-pre-processing"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198954    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" containerName="payload-pre-processing"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198979    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0db5baa-2b3e-4920-b72d-6a17a70843ba" containerName="main"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.198987    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0db5baa-2b3e-4920-b72d-6a17a70843ba" containerName="main"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.199047    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f10aff-5699-4147-aa25-7e9326a1757c" containerName="payload-processing"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.199064    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="afec4335-6dc4-4f6e-bd54-f1a0a00ff253" containerName="maas-api"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.199072    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f0bfc9-c7a8-4a7d-a4e4-3856165b9098" containerName="payload-pre-processing"
Sep 14 08:09:14.199109 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.199080    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0db5baa-2b3e-4920-b72d-6a17a70843ba" containerName="main"
Sep 14 08:09:14.202299 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.202277    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:14.204518 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.204496    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qcf9c\""
Sep 14 08:09:14.207733 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.207712    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-7bda51"]
Sep 14 08:09:14.304920 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.304892    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kqkx\" (UniqueName: \"kubernetes.io/projected/8261049b-229d-4d2d-b4ec-eea417a1d130-kube-api-access-7kqkx\") pod \"test-curl-7749-7bda51\" (UID: \"8261049b-229d-4d2d-b4ec-eea417a1d130\") " pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:14.406332 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.406294    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kqkx\" (UniqueName: \"kubernetes.io/projected/8261049b-229d-4d2d-b4ec-eea417a1d130-kube-api-access-7kqkx\") pod \"test-curl-7749-7bda51\" (UID: \"8261049b-229d-4d2d-b4ec-eea417a1d130\") " pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:14.413911 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.413877    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kqkx\" (UniqueName: \"kubernetes.io/projected/8261049b-229d-4d2d-b4ec-eea417a1d130-kube-api-access-7kqkx\") pod \"test-curl-7749-7bda51\" (UID: \"8261049b-229d-4d2d-b4ec-eea417a1d130\") " pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:14.513809 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.513720    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:14.785001 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.784953    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-7bda51"]
Sep 14 08:09:14.786654 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:09:14.786628    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8261049b_229d_4d2d_b4ec_eea417a1d130.slice/crio-117a689a66f8cb8806836dc2e25b91b65cf252b97673b0420edd5a8d2279cd72 WatchSource:0}: Error finding container 117a689a66f8cb8806836dc2e25b91b65cf252b97673b0420edd5a8d2279cd72: Status 404 returned error can't find the container with id 117a689a66f8cb8806836dc2e25b91b65cf252b97673b0420edd5a8d2279cd72
Sep 14 08:09:14.788647 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:14.788629    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:09:15.087087 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:15.087001    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-7bda51" event={"ID":"8261049b-229d-4d2d-b4ec-eea417a1d130","Type":"ContainerStarted","Data":"117a689a66f8cb8806836dc2e25b91b65cf252b97673b0420edd5a8d2279cd72"}
Sep 14 08:09:16.094761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.094721    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-7bda51" event={"ID":"8261049b-229d-4d2d-b4ec-eea417a1d130","Type":"ContainerStarted","Data":"c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4"}
Sep 14 08:09:16.112434 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.112353    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-7bda51" podStartSLOduration=1.238670784 podStartE2EDuration="2.112337823s" totalImagesPullingTime="873.667039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:16.109845406 +0000 UTC m=+1212.609238197" watchObservedRunningTime="2026-09-14 08:09:16.112337823 +0000 UTC m=+1212.611730601"
Sep 14 08:09:16.720401 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.720359    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-7bda51"]
Sep 14 08:09:16.724491 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.724454    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-7bda51"]
Sep 14 08:09:16.830406 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.830325    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-2997e6"]
Sep 14 08:09:16.830784 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.830766    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8261049b-229d-4d2d-b4ec-eea417a1d130" containerName="test-curl-7749-7bda51"
Sep 14 08:09:16.830870 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.830787    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8261049b-229d-4d2d-b4ec-eea417a1d130" containerName="test-curl-7749-7bda51"
Sep 14 08:09:16.830925 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.830900    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8261049b-229d-4d2d-b4ec-eea417a1d130" containerName="test-curl-7749-7bda51"
Sep 14 08:09:16.834127 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.834108    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:16.839203 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.839179    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-2997e6"]
Sep 14 08:09:16.931194 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:16.931149    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55h84\" (UniqueName: \"kubernetes.io/projected/1d47d54f-7006-4e04-b8cd-214f4b6d000c-kube-api-access-55h84\") pod \"test-curl-7749-2997e6\" (UID: \"1d47d54f-7006-4e04-b8cd-214f4b6d000c\") " pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:17.032159 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:17.032117    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55h84\" (UniqueName: \"kubernetes.io/projected/1d47d54f-7006-4e04-b8cd-214f4b6d000c-kube-api-access-55h84\") pod \"test-curl-7749-2997e6\" (UID: \"1d47d54f-7006-4e04-b8cd-214f4b6d000c\") " pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:17.039073 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:17.039036    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55h84\" (UniqueName: \"kubernetes.io/projected/1d47d54f-7006-4e04-b8cd-214f4b6d000c-kube-api-access-55h84\") pod \"test-curl-7749-2997e6\" (UID: \"1d47d54f-7006-4e04-b8cd-214f4b6d000c\") " pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:17.100328 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:17.100253    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-7bda51" container="test-curl-7749-7bda51"
Sep 14 08:09:17.144901 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:17.144871    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:17.414431 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:17.414405    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-2997e6"]
Sep 14 08:09:17.416058 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:09:17.416022    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d47d54f_7006_4e04_b8cd_214f4b6d000c.slice/crio-d86c5292a12cc5b8d7a2a124e4f45aa7447809550740b8cc47432477fc302aea WatchSource:0}: Error finding container d86c5292a12cc5b8d7a2a124e4f45aa7447809550740b8cc47432477fc302aea: Status 404 returned error can't find the container with id d86c5292a12cc5b8d7a2a124e4f45aa7447809550740b8cc47432477fc302aea
Sep 14 08:09:18.108163 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:18.108115    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-2997e6" event={"ID":"1d47d54f-7006-4e04-b8cd-214f4b6d000c","Type":"ContainerStarted","Data":"03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43"}
Sep 14 08:09:18.108594 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:18.108171    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-2997e6" event={"ID":"1d47d54f-7006-4e04-b8cd-214f4b6d000c","Type":"ContainerStarted","Data":"d86c5292a12cc5b8d7a2a124e4f45aa7447809550740b8cc47432477fc302aea"}
Sep 14 08:09:18.108594 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:18.108193    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-7bda51" podUID="8261049b-229d-4d2d-b4ec-eea417a1d130" containerName="test-curl-7749-7bda51" containerID="cri-o://c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4" gracePeriod=2
Sep 14 08:09:18.121725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:18.121678    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-2997e6" podStartSLOduration=2.121660135 podStartE2EDuration="2.121660135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:18.120438584 +0000 UTC m=+1214.619831362" watchObservedRunningTime="2026-09-14 08:09:18.121660135 +0000 UTC m=+1214.621052912"
Sep 14 08:09:18.707112 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:18.707066    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-2997e6"]
Sep 14 08:09:18.710534 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:18.710492    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-2997e6"]
Sep 14 08:09:19.113383 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:19.113342    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-2997e6" container="test-curl-7749-2997e6"
Sep 14 08:09:20.119820 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:20.119783    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-2997e6" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" containerName="test-curl-7749-2997e6" containerID="cri-o://03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43" gracePeriod=2
Sep 14 08:09:20.295944 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:20.295920    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:20.363886 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:20.363800    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kqkx\" (UniqueName: \"kubernetes.io/projected/8261049b-229d-4d2d-b4ec-eea417a1d130-kube-api-access-7kqkx\") pod \"8261049b-229d-4d2d-b4ec-eea417a1d130\" (UID: \"8261049b-229d-4d2d-b4ec-eea417a1d130\") "
Sep 14 08:09:20.365767 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:20.365744    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8261049b-229d-4d2d-b4ec-eea417a1d130-kube-api-access-7kqkx" (OuterVolumeSpecName: "kube-api-access-7kqkx") pod "8261049b-229d-4d2d-b4ec-eea417a1d130" (UID: "8261049b-229d-4d2d-b4ec-eea417a1d130"). InnerVolumeSpecName "kube-api-access-7kqkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:20.465323 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:20.465286    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kqkx\" (UniqueName: \"kubernetes.io/projected/8261049b-229d-4d2d-b4ec-eea417a1d130-kube-api-access-7kqkx\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:21.125655 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:21.125616    2556 generic.go:356] "Generic (PLEG): container finished" podID="8261049b-229d-4d2d-b4ec-eea417a1d130" containerID="c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4" exitCode=137
Sep 14 08:09:21.126121 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:21.125666    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:21.126121 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:21.125677    2556 scope.go:117] "RemoveContainer" containerID="c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4"
Sep 14 08:09:21.134561 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:21.134543    2556 scope.go:117] "RemoveContainer" containerID="c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4"
Sep 14 08:09:21.134836 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:21.134817    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4\": container with ID starting with c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4 not found: ID does not exist" containerID="c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4"
Sep 14 08:09:21.134903 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:21.134845    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4"} err="failed to get container status \"c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4\": rpc error: code = NotFound desc = could not find container \"c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4\": container with ID starting with c16b9a329c21f558fe1b42f0f0cf9a0afa89cfe84700f97c0e0eb82b38522fa4 not found: ID does not exist"
Sep 14 08:09:21.135672 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:21.135649    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-7bda51\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="8261049b-229d-4d2d-b4ec-eea417a1d130" pod="opendatahub/test-curl-7749-7bda51"
Sep 14 08:09:22.018894 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:22.018848    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8261049b-229d-4d2d-b4ec-eea417a1d130" path="/var/lib/kubelet/pods/8261049b-229d-4d2d-b4ec-eea417a1d130/volumes"
Sep 14 08:09:22.294628 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:22.294593    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:22.383781 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:22.383743    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55h84\" (UniqueName: \"kubernetes.io/projected/1d47d54f-7006-4e04-b8cd-214f4b6d000c-kube-api-access-55h84\") pod \"1d47d54f-7006-4e04-b8cd-214f4b6d000c\" (UID: \"1d47d54f-7006-4e04-b8cd-214f4b6d000c\") "
Sep 14 08:09:22.385733 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:22.385704    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d47d54f-7006-4e04-b8cd-214f4b6d000c-kube-api-access-55h84" (OuterVolumeSpecName: "kube-api-access-55h84") pod "1d47d54f-7006-4e04-b8cd-214f4b6d000c" (UID: "1d47d54f-7006-4e04-b8cd-214f4b6d000c"). InnerVolumeSpecName "kube-api-access-55h84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:22.485106 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:22.485080    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55h84\" (UniqueName: \"kubernetes.io/projected/1d47d54f-7006-4e04-b8cd-214f4b6d000c-kube-api-access-55h84\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.141342 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:23.141304    2556 generic.go:356] "Generic (PLEG): container finished" podID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" containerID="03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43" exitCode=137
Sep 14 08:09:23.141648 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:23.141346    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:23.141648 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:23.141401    2556 scope.go:117] "RemoveContainer" containerID="03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43"
Sep 14 08:09:23.152043 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:23.152022    2556 scope.go:117] "RemoveContainer" containerID="03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43"
Sep 14 08:09:23.152570 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:23.152347    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43\": container with ID starting with 03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43 not found: ID does not exist" containerID="03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43"
Sep 14 08:09:23.152570 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:23.152381    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43"} err="failed to get container status \"03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43\": rpc error: code = NotFound desc = could not find container \"03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43\": container with ID starting with 03bd24260db542bd2ce8122b2ec6659a38de0ec42741ac0b15b953cbe65e2c43 not found: ID does not exist"
Sep 14 08:09:23.154602 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:23.154575    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-2997e6\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:24.017581 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:09:24.017542    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-2997e6\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" pod="opendatahub/test-curl-7749-2997e6"
Sep 14 08:09:24.019751 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:24.019727    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" path="/var/lib/kubelet/pods/1d47d54f-7006-4e04-b8cd-214f4b6d000c/volumes"
Sep 14 08:09:32.047334 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.047292    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"]
Sep 14 08:09:32.047871 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.047850    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" containerName="test-curl-7749-2997e6"
Sep 14 08:09:32.047951 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.047874    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" containerName="test-curl-7749-2997e6"
Sep 14 08:09:32.048036 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.047998    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d47d54f-7006-4e04-b8cd-214f4b6d000c" containerName="test-curl-7749-2997e6"
Sep 14 08:09:32.052439 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.052416    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.054645 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.054622    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2a94f0ce-serving-cert\""
Sep 14 08:09:32.060984 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.060219    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"]
Sep 14 08:09:32.171334 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.171295    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6824\" (UniqueName: \"kubernetes.io/projected/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-kube-api-access-n6824\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.171334 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.171337    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-maas-api-tls\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.171566 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.171360    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-openshift-service-ca\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.171566 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.171422    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-metrics-certs\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.272375 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.272345    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-openshift-service-ca\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.272375 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.272383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-metrics-certs\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.272584 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.272460    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6824\" (UniqueName: \"kubernetes.io/projected/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-kube-api-access-n6824\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.272584 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.272479    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-maas-api-tls\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.273108 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.273084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-openshift-service-ca\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.274701 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.274667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-metrics-certs\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.274802 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.274713    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-maas-api-tls\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.283881 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.283857    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6824\" (UniqueName: \"kubernetes.io/projected/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-kube-api-access-n6824\") pod \"maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.365236 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.365153    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:32.635003 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.634895    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"]
Sep 14 08:09:32.639126 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.639101    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.641254 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.641223    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"]
Sep 14 08:09:32.641370 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.641278    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2a94f0ce-dockercfg-xw5cm\""
Sep 14 08:09:32.641432 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.641417    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2a94f0ce\""
Sep 14 08:09:32.642886 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:09:32.642861    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb1fe83_dde4_48d1_ab16_e1317ce9e03c.slice/crio-533e02c04a124b55815b4ade299876eabafd02776eb9cf4f4ac5cbdce8dc62c6 WatchSource:0}: Error finding container 533e02c04a124b55815b4ade299876eabafd02776eb9cf4f4ac5cbdce8dc62c6: Status 404 returned error can't find the container with id 533e02c04a124b55815b4ade299876eabafd02776eb9cf4f4ac5cbdce8dc62c6
Sep 14 08:09:32.649398 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.649377    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"]
Sep 14 08:09:32.673676 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.673644    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"]
Sep 14 08:09:32.677924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.677905    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.684986 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.684941    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"]
Sep 14 08:09:32.776881 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.776839    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/578a96a0-86f6-4b96-9056-a3817c689385-plugins-config-volume\") pod \"payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") " pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.777064 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.776897    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bj52\" (UniqueName: \"kubernetes.io/projected/ddd576e4-0713-463a-a06c-02f0366d36cb-kube-api-access-7bj52\") pod \"payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.777064 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.776995    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd576e4-0713-463a-a06c-02f0366d36cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.777064 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.777040    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wblmc\" (UniqueName: \"kubernetes.io/projected/578a96a0-86f6-4b96-9056-a3817c689385-kube-api-access-wblmc\") pod \"payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") " pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.878185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.878152    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/578a96a0-86f6-4b96-9056-a3817c689385-plugins-config-volume\") pod \"payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") " pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.878329 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.878195    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bj52\" (UniqueName: \"kubernetes.io/projected/ddd576e4-0713-463a-a06c-02f0366d36cb-kube-api-access-7bj52\") pod \"payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.878329 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.878250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd576e4-0713-463a-a06c-02f0366d36cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.878329 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.878267    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wblmc\" (UniqueName: \"kubernetes.io/projected/578a96a0-86f6-4b96-9056-a3817c689385-kube-api-access-wblmc\") pod \"payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") " pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.879147 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.879122    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd576e4-0713-463a-a06c-02f0366d36cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.879213 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.879122    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/578a96a0-86f6-4b96-9056-a3817c689385-plugins-config-volume\") pod \"payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") " pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.892924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.892852    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bj52\" (UniqueName: \"kubernetes.io/projected/ddd576e4-0713-463a-a06c-02f0366d36cb-kube-api-access-7bj52\") pod \"payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.892924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.892907    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wblmc\" (UniqueName: \"kubernetes.io/projected/578a96a0-86f6-4b96-9056-a3817c689385-kube-api-access-wblmc\") pod \"payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") " pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:32.952418 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.952387    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:32.990622 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:32.990589    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:33.200827 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:33.200792    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2" event={"ID":"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c","Type":"ContainerStarted","Data":"c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16"}
Sep 14 08:09:33.201241 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:33.200855    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2" event={"ID":"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c","Type":"ContainerStarted","Data":"533e02c04a124b55815b4ade299876eabafd02776eb9cf4f4ac5cbdce8dc62c6"}
Sep 14 08:09:33.201241 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:33.200987    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:33.221447 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:33.221404    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2" podStartSLOduration=0.784278085 podStartE2EDuration="1.221390519s" totalImagesPullingTime="437.112434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:33.22029753 +0000 UTC m=+1229.719690328" watchObservedRunningTime="2026-09-14 08:09:33.221390519 +0000 UTC m=+1229.720783296"
Sep 14 08:09:33.226077 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:33.226058    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"]
Sep 14 08:09:33.228085 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:09:33.228045    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd576e4_0713_463a_a06c_02f0366d36cb.slice/crio-49bc55f98824f5c5e20cf72dd13737e81809a5766f8f8dbf9a6775132acfb94a WatchSource:0}: Error finding container 49bc55f98824f5c5e20cf72dd13737e81809a5766f8f8dbf9a6775132acfb94a: Status 404 returned error can't find the container with id 49bc55f98824f5c5e20cf72dd13737e81809a5766f8f8dbf9a6775132acfb94a
Sep 14 08:09:33.273465 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:33.273435    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"]
Sep 14 08:09:33.275835 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:09:33.275776    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578a96a0_86f6_4b96_9056_a3817c689385.slice/crio-a9260190103549ea5695a13aea265d0beb8cd5ebc88e584b9075756e32577e05 WatchSource:0}: Error finding container a9260190103549ea5695a13aea265d0beb8cd5ebc88e584b9075756e32577e05: Status 404 returned error can't find the container with id a9260190103549ea5695a13aea265d0beb8cd5ebc88e584b9075756e32577e05
Sep 14 08:09:34.208433 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.208393    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl" event={"ID":"578a96a0-86f6-4b96-9056-a3817c689385","Type":"ContainerStarted","Data":"bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62"}
Sep 14 08:09:34.208932 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.208459    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl" event={"ID":"578a96a0-86f6-4b96-9056-a3817c689385","Type":"ContainerStarted","Data":"a9260190103549ea5695a13aea265d0beb8cd5ebc88e584b9075756e32577e05"}
Sep 14 08:09:34.208932 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.208591    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:34.210259 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.210232    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz" event={"ID":"ddd576e4-0713-463a-a06c-02f0366d36cb","Type":"ContainerStarted","Data":"9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70"}
Sep 14 08:09:34.210402 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.210274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz" event={"ID":"ddd576e4-0713-463a-a06c-02f0366d36cb","Type":"ContainerStarted","Data":"49bc55f98824f5c5e20cf72dd13737e81809a5766f8f8dbf9a6775132acfb94a"}
Sep 14 08:09:34.210402 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.210303    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:34.224364 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.224317    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl" podStartSLOduration=2.224297957 podStartE2EDuration="2.224297957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:34.223917412 +0000 UTC m=+1230.723310207" watchObservedRunningTime="2026-09-14 08:09:34.224297957 +0000 UTC m=+1230.723690734"
Sep 14 08:09:34.239801 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:34.239742    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz" podStartSLOduration=2.239720912 podStartE2EDuration="2.239720912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:34.238677687 +0000 UTC m=+1230.738070469" watchObservedRunningTime="2026-09-14 08:09:34.239720912 +0000 UTC m=+1230.739113691"
Sep 14 08:09:39.215910 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:39.215877    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:09:40.334544 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.334507    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"]
Sep 14 08:09:40.338279 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.338261    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.340292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.340264    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-919cc9\""
Sep 14 08:09:40.340292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.340264    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-919cc9-dockercfg-tbcfv\""
Sep 14 08:09:40.347634 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.347612    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"]
Sep 14 08:09:40.451709 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.451680    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.451857 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.451715    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tdh4\" (UniqueName: \"kubernetes.io/projected/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-kube-api-access-9tdh4\") pod \"payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.553110 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.553070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.553110 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.553112    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tdh4\" (UniqueName: \"kubernetes.io/projected/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-kube-api-access-9tdh4\") pod \"payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.553788 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.553767    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.560448 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.560421    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tdh4\" (UniqueName: \"kubernetes.io/projected/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-kube-api-access-9tdh4\") pod \"payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.648982 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.648866    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:40.920869 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:40.920842    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"]
Sep 14 08:09:40.922210 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:09:40.922183    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bd445e_f35f_47de_8e52_e806d9c3e4c8.slice/crio-a8af011d72ca9b84dcc176290e4195840a2afbb10c8e88c292d0834efefff5d1 WatchSource:0}: Error finding container a8af011d72ca9b84dcc176290e4195840a2afbb10c8e88c292d0834efefff5d1: Status 404 returned error can't find the container with id a8af011d72ca9b84dcc176290e4195840a2afbb10c8e88c292d0834efefff5d1
Sep 14 08:09:41.256846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:41.256738    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh" event={"ID":"e0bd445e-f35f-47de-8e52-e806d9c3e4c8","Type":"ContainerStarted","Data":"2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0"}
Sep 14 08:09:41.256846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:41.256815    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh" event={"ID":"e0bd445e-f35f-47de-8e52-e806d9c3e4c8","Type":"ContainerStarted","Data":"a8af011d72ca9b84dcc176290e4195840a2afbb10c8e88c292d0834efefff5d1"}
Sep 14 08:09:41.256846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:41.256847    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:09:41.275627 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:41.275561    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh" podStartSLOduration=1.275541055 podStartE2EDuration="1.275541055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:41.27183309 +0000 UTC m=+1237.771225894" watchObservedRunningTime="2026-09-14 08:09:41.275541055 +0000 UTC m=+1237.774933834"
Sep 14 08:09:45.218313 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:45.218278    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:09:45.218829 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:45.218474    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:09:52.264712 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:09:52.264680    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:10:14.560606 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.560559    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"]
Sep 14 08:10:14.565677 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.565658    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.567748 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.567726    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"distinct-2-w0-831f8e-kserve-self-signed-certs\""
Sep 14 08:10:14.574744 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.574720    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"]
Sep 14 08:10:14.675837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.675805    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-home\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.675837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.675850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tls-certs\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.676095 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.675870    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dpgz\" (UniqueName: \"kubernetes.io/projected/1ef42a8e-fda2-4676-b394-6c3e92755b9f-kube-api-access-7dpgz\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.676095 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.675947    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tmp-dir\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.676095 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.676055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-model-cache\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.676095 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.676093    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-dshm\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777241 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777208    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-model-cache\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777257    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-dshm\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777310    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-home\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tls-certs\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dpgz\" (UniqueName: \"kubernetes.io/projected/1ef42a8e-fda2-4676-b394-6c3e92755b9f-kube-api-access-7dpgz\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tmp-dir\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777680 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777663    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-model-cache\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777795 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777772    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tmp-dir\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.777887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.777868    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-home\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.779644 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.779622    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-dshm\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.779934 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.779911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tls-certs\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.784583 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.784557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dpgz\" (UniqueName: \"kubernetes.io/projected/1ef42a8e-fda2-4676-b394-6c3e92755b9f-kube-api-access-7dpgz\") pod \"distinct-2-w0-831f8e-kserve-64d4744589-z6jhs\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") " pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:14.875725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:14.875647    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:15.147857 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:15.147817    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"]
Sep 14 08:10:15.149030 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:10:15.149004    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ef42a8e_fda2_4676_b394_6c3e92755b9f.slice/crio-b9627e8713111afc31ef04c9356f47ee1bbff130597cecbbef385088b501d38d WatchSource:0}: Error finding container b9627e8713111afc31ef04c9356f47ee1bbff130597cecbbef385088b501d38d: Status 404 returned error can't find the container with id b9627e8713111afc31ef04c9356f47ee1bbff130597cecbbef385088b501d38d
Sep 14 08:10:15.469455 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:15.469413    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs" event={"ID":"1ef42a8e-fda2-4676-b394-6c3e92755b9f","Type":"ContainerStarted","Data":"1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3"}
Sep 14 08:10:15.469647 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:15.469463    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs" event={"ID":"1ef42a8e-fda2-4676-b394-6c3e92755b9f","Type":"ContainerStarted","Data":"b9627e8713111afc31ef04c9356f47ee1bbff130597cecbbef385088b501d38d"}
Sep 14 08:10:15.485214 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:15.485167    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs" podStartSLOduration=1.485151623 podStartE2EDuration="1.485151623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:10:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:10:15.483414855 +0000 UTC m=+1271.982807645" watchObservedRunningTime="2026-09-14 08:10:15.485151623 +0000 UTC m=+1271.984544400"
Sep 14 08:10:18.762977 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:18.762916    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"]
Sep 14 08:10:24.875902 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:24.875865    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:24.876371 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:24.876008    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:24.889069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:24.889045    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:25.535774 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:25.535740    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:10:33.783556 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:33.783515    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4" podUID="f56c2a94-430b-46e1-be42-9e112fd09fa4" containerName="main" containerID="cri-o://81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5" gracePeriod=45
Sep 14 08:10:34.033992 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.033924    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:10:34.163682 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.163650    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq8tj\" (UniqueName: \"kubernetes.io/projected/f56c2a94-430b-46e1-be42-9e112fd09fa4-kube-api-access-mq8tj\") pod \"f56c2a94-430b-46e1-be42-9e112fd09fa4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") "
Sep 14 08:10:34.163682 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.163689    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56c2a94-430b-46e1-be42-9e112fd09fa4-tls-certs\") pod \"f56c2a94-430b-46e1-be42-9e112fd09fa4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") "
Sep 14 08:10:34.163989 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.163712    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-tmp-dir\") pod \"f56c2a94-430b-46e1-be42-9e112fd09fa4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") "
Sep 14 08:10:34.163989 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.163758    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-dshm\") pod \"f56c2a94-430b-46e1-be42-9e112fd09fa4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") "
Sep 14 08:10:34.163989 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.163800    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-model-cache\") pod \"f56c2a94-430b-46e1-be42-9e112fd09fa4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") "
Sep 14 08:10:34.163989 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.163829    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-home\") pod \"f56c2a94-430b-46e1-be42-9e112fd09fa4\" (UID: \"f56c2a94-430b-46e1-be42-9e112fd09fa4\") "
Sep 14 08:10:34.164218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.164058    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f56c2a94-430b-46e1-be42-9e112fd09fa4" (UID: "f56c2a94-430b-46e1-be42-9e112fd09fa4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:34.164218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.164142    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-model-cache" (OuterVolumeSpecName: "model-cache") pod "f56c2a94-430b-46e1-be42-9e112fd09fa4" (UID: "f56c2a94-430b-46e1-be42-9e112fd09fa4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:34.164218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.164169    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:10:34.164333 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.164206    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-home" (OuterVolumeSpecName: "home") pod "f56c2a94-430b-46e1-be42-9e112fd09fa4" (UID: "f56c2a94-430b-46e1-be42-9e112fd09fa4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:34.166008 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.165981    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f56c2a94-430b-46e1-be42-9e112fd09fa4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f56c2a94-430b-46e1-be42-9e112fd09fa4" (UID: "f56c2a94-430b-46e1-be42-9e112fd09fa4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:10:34.169735 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.166466    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-dshm" (OuterVolumeSpecName: "dshm") pod "f56c2a94-430b-46e1-be42-9e112fd09fa4" (UID: "f56c2a94-430b-46e1-be42-9e112fd09fa4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:34.169735 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.166074    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56c2a94-430b-46e1-be42-9e112fd09fa4-kube-api-access-mq8tj" (OuterVolumeSpecName: "kube-api-access-mq8tj") pod "f56c2a94-430b-46e1-be42-9e112fd09fa4" (UID: "f56c2a94-430b-46e1-be42-9e112fd09fa4"). InnerVolumeSpecName "kube-api-access-mq8tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:10:34.264761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.264732    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq8tj\" (UniqueName: \"kubernetes.io/projected/f56c2a94-430b-46e1-be42-9e112fd09fa4-kube-api-access-mq8tj\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:10:34.264761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.264757    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56c2a94-430b-46e1-be42-9e112fd09fa4-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:10:34.264761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.264766    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:10:34.264982 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.264775    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:10:34.264982 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.264782    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56c2a94-430b-46e1-be42-9e112fd09fa4-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:10:34.576854 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.576822    2556 generic.go:356] "Generic (PLEG): container finished" podID="f56c2a94-430b-46e1-be42-9e112fd09fa4" containerID="81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5" exitCode=0
Sep 14 08:10:34.577038 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.576879    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"
Sep 14 08:10:34.577038 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.576905    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4" event={"ID":"f56c2a94-430b-46e1-be42-9e112fd09fa4","Type":"ContainerDied","Data":"81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5"}
Sep 14 08:10:34.577038 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.576953    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4" event={"ID":"f56c2a94-430b-46e1-be42-9e112fd09fa4","Type":"ContainerDied","Data":"8b7129e04c93ca387e929aefed3bc75b7c602bc7675b89f6e2718ab2d2637b06"}
Sep 14 08:10:34.577038 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.576986    2556 scope.go:117] "RemoveContainer" containerID="81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5"
Sep 14 08:10:34.586752 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.586733    2556 scope.go:117] "RemoveContainer" containerID="81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5"
Sep 14 08:10:34.587053 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:10:34.587033    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5\": container with ID starting with 81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5 not found: ID does not exist" containerID="81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5"
Sep 14 08:10:34.587122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.587064    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5"} err="failed to get container status \"81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5\": rpc error: code = NotFound desc = could not find container \"81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5\": container with ID starting with 81b2283ac833c656f87fddca86cdf4e98c07e30913a3686fbd25002b4b5971c5 not found: ID does not exist"
Sep 14 08:10:34.597912 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.597890    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"]
Sep 14 08:10:34.603185 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:34.603166    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/e2e-conflict-b-2827d38c-kserve-76d47b5c6-fqdb4"]
Sep 14 08:10:36.019532 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:10:36.019496    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f56c2a94-430b-46e1-be42-9e112fd09fa4" path="/var/lib/kubelet/pods/f56c2a94-430b-46e1-be42-9e112fd09fa4/volumes"
Sep 14 08:11:58.475690 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.475650    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"]
Sep 14 08:11:58.476187 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.476066    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56c2a94-430b-46e1-be42-9e112fd09fa4" containerName="main"
Sep 14 08:11:58.476187 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.476078    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56c2a94-430b-46e1-be42-9e112fd09fa4" containerName="main"
Sep 14 08:11:58.476187 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.476166    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56c2a94-430b-46e1-be42-9e112fd09fa4" containerName="main"
Sep 14 08:11:58.479356 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.479330    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.481650 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.481632    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"distinct-w4-cd7582-kserve-self-signed-certs\""
Sep 14 08:11:58.491241 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.491220    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"]
Sep 14 08:11:58.570042 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.570002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-model-cache\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.570218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.570066    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz578\" (UniqueName: \"kubernetes.io/projected/2c39f433-5f93-487d-8035-74738268eb47-kube-api-access-gz578\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.570218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.570135    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-dshm\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.570218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.570189    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-tmp-dir\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.570218 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.570209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c39f433-5f93-487d-8035-74738268eb47-tls-certs\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.570394 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.570246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-home\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.670898 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.670855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-model-cache\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671135 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.670930    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz578\" (UniqueName: \"kubernetes.io/projected/2c39f433-5f93-487d-8035-74738268eb47-kube-api-access-gz578\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671135 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.670998    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-dshm\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671135 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.671032    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-tmp-dir\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671135 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.671053    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c39f433-5f93-487d-8035-74738268eb47-tls-certs\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671135 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.671105    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-home\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671513 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.671478    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-model-cache\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671638 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.671501    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-tmp-dir\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.671638 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.671582    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-home\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.673404 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.673374    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-dshm\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.673618 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.673601    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c39f433-5f93-487d-8035-74738268eb47-tls-certs\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.678072 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.678051    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz578\" (UniqueName: \"kubernetes.io/projected/2c39f433-5f93-487d-8035-74738268eb47-kube-api-access-gz578\") pod \"distinct-w4-cd7582-kserve-874ccd59d-cn54n\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:58.788390 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:58.788301    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:11:59.061136 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:11:59.061108    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"]
Sep 14 08:11:59.062453 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:11:59.062423    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c39f433_5f93_487d_8035_74738268eb47.slice/crio-b605e181761f3fa76acdf48f0cdc598181690cae42a61b111d3e914ad7f514f7 WatchSource:0}: Error finding container b605e181761f3fa76acdf48f0cdc598181690cae42a61b111d3e914ad7f514f7: Status 404 returned error can't find the container with id b605e181761f3fa76acdf48f0cdc598181690cae42a61b111d3e914ad7f514f7
Sep 14 08:12:00.061127 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:00.061086    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n" event={"ID":"2c39f433-5f93-487d-8035-74738268eb47","Type":"ContainerStarted","Data":"b8d5a8bbd1f4287cdd75d6f12bb44f12e7ac52c967a062aff74983d53fef1c13"}
Sep 14 08:12:00.061574 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:00.061140    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n" event={"ID":"2c39f433-5f93-487d-8035-74738268eb47","Type":"ContainerStarted","Data":"b605e181761f3fa76acdf48f0cdc598181690cae42a61b111d3e914ad7f514f7"}
Sep 14 08:12:00.078081 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:00.078025    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n" podStartSLOduration=2.078008933 podStartE2EDuration="2.078008933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:12:00.075469454 +0000 UTC m=+1376.574862253" watchObservedRunningTime="2026-09-14 08:12:00.078008933 +0000 UTC m=+1376.577401710"
Sep 14 08:12:01.525766 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.525729    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"]
Sep 14 08:12:01.526255 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.525997    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2" podUID="5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" containerName="maas-api" containerID="cri-o://c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16" gracePeriod=30
Sep 14 08:12:01.576697 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.576660    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"]
Sep 14 08:12:01.576952 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.576924    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl" podUID="578a96a0-86f6-4b96-9056-a3817c689385" containerName="payload-processing" containerID="cri-o://bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62" gracePeriod=30
Sep 14 08:12:01.583261 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.583229    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"]
Sep 14 08:12:01.583441 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.583413    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz" podUID="ddd576e4-0713-463a-a06c-02f0366d36cb" containerName="payload-pre-processing" containerID="cri-o://9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70" gracePeriod=30
Sep 14 08:12:01.845851 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.845821    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:12:01.886204 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.886173    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:12:01.913625 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:01.913597    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:12:02.003588 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003555    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wblmc\" (UniqueName: \"kubernetes.io/projected/578a96a0-86f6-4b96-9056-a3817c689385-kube-api-access-wblmc\") pod \"578a96a0-86f6-4b96-9056-a3817c689385\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") "
Sep 14 08:12:02.003750 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003596    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/578a96a0-86f6-4b96-9056-a3817c689385-plugins-config-volume\") pod \"578a96a0-86f6-4b96-9056-a3817c689385\" (UID: \"578a96a0-86f6-4b96-9056-a3817c689385\") "
Sep 14 08:12:02.003750 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003647    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-metrics-certs\") pod \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") "
Sep 14 08:12:02.003750 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003676    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6824\" (UniqueName: \"kubernetes.io/projected/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-kube-api-access-n6824\") pod \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") "
Sep 14 08:12:02.003878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003751    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-maas-api-tls\") pod \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") "
Sep 14 08:12:02.003878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003770    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd576e4-0713-463a-a06c-02f0366d36cb-plugins-config-volume\") pod \"ddd576e4-0713-463a-a06c-02f0366d36cb\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") "
Sep 14 08:12:02.003878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bj52\" (UniqueName: \"kubernetes.io/projected/ddd576e4-0713-463a-a06c-02f0366d36cb-kube-api-access-7bj52\") pod \"ddd576e4-0713-463a-a06c-02f0366d36cb\" (UID: \"ddd576e4-0713-463a-a06c-02f0366d36cb\") "
Sep 14 08:12:02.003878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.003836    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-openshift-service-ca\") pod \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\" (UID: \"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c\") "
Sep 14 08:12:02.004462 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.004180    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/578a96a0-86f6-4b96-9056-a3817c689385-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "578a96a0-86f6-4b96-9056-a3817c689385" (UID: "578a96a0-86f6-4b96-9056-a3817c689385"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:02.004462 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.004404    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddd576e4-0713-463a-a06c-02f0366d36cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddd576e4-0713-463a-a06c-02f0366d36cb" (UID: "ddd576e4-0713-463a-a06c-02f0366d36cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:02.004462 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.004440    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" (UID: "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:02.005892 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.005861    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" (UID: "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:02.006028 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.005934    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" (UID: "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:02.006124 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.006097    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd576e4-0713-463a-a06c-02f0366d36cb-kube-api-access-7bj52" (OuterVolumeSpecName: "kube-api-access-7bj52") pod "ddd576e4-0713-463a-a06c-02f0366d36cb" (UID: "ddd576e4-0713-463a-a06c-02f0366d36cb"). InnerVolumeSpecName "kube-api-access-7bj52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:02.006276 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.006254    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-kube-api-access-n6824" (OuterVolumeSpecName: "kube-api-access-n6824") pod "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" (UID: "5fb1fe83-dde4-48d1-ab16-e1317ce9e03c"). InnerVolumeSpecName "kube-api-access-n6824". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:02.006360 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.006262    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/578a96a0-86f6-4b96-9056-a3817c689385-kube-api-access-wblmc" (OuterVolumeSpecName: "kube-api-access-wblmc") pod "578a96a0-86f6-4b96-9056-a3817c689385" (UID: "578a96a0-86f6-4b96-9056-a3817c689385"). InnerVolumeSpecName "kube-api-access-wblmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:02.075736 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.075665    2556 generic.go:356] "Generic (PLEG): container finished" podID="5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" containerID="c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16" exitCode=0
Sep 14 08:12:02.075864 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.075735    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"
Sep 14 08:12:02.075864 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.075759    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2" event={"ID":"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c","Type":"ContainerDied","Data":"c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16"}
Sep 14 08:12:02.075864 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.075817    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2" event={"ID":"5fb1fe83-dde4-48d1-ab16-e1317ce9e03c","Type":"ContainerDied","Data":"533e02c04a124b55815b4ade299876eabafd02776eb9cf4f4ac5cbdce8dc62c6"}
Sep 14 08:12:02.075864 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.075847    2556 scope.go:117] "RemoveContainer" containerID="c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16"
Sep 14 08:12:02.077774 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.077715    2556 generic.go:356] "Generic (PLEG): container finished" podID="578a96a0-86f6-4b96-9056-a3817c689385" containerID="bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62" exitCode=0
Sep 14 08:12:02.077880 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.077797    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl" event={"ID":"578a96a0-86f6-4b96-9056-a3817c689385","Type":"ContainerDied","Data":"bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62"}
Sep 14 08:12:02.077880 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.077851    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl" event={"ID":"578a96a0-86f6-4b96-9056-a3817c689385","Type":"ContainerDied","Data":"a9260190103549ea5695a13aea265d0beb8cd5ebc88e584b9075756e32577e05"}
Sep 14 08:12:02.078066 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.077886    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"
Sep 14 08:12:02.079589 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.079570    2556 generic.go:356] "Generic (PLEG): container finished" podID="ddd576e4-0713-463a-a06c-02f0366d36cb" containerID="9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70" exitCode=0
Sep 14 08:12:02.079696 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.079644    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz" event={"ID":"ddd576e4-0713-463a-a06c-02f0366d36cb","Type":"ContainerDied","Data":"9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70"}
Sep 14 08:12:02.079696 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.079683    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz" event={"ID":"ddd576e4-0713-463a-a06c-02f0366d36cb","Type":"ContainerDied","Data":"49bc55f98824f5c5e20cf72dd13737e81809a5766f8f8dbf9a6775132acfb94a"}
Sep 14 08:12:02.079818 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.079744    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"
Sep 14 08:12:02.087107 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.087087    2556 scope.go:117] "RemoveContainer" containerID="c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16"
Sep 14 08:12:02.087377 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:12:02.087357    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16\": container with ID starting with c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16 not found: ID does not exist" containerID="c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16"
Sep 14 08:12:02.087432 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.087385    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16"} err="failed to get container status \"c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16\": rpc error: code = NotFound desc = could not find container \"c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16\": container with ID starting with c079f478d84c8a75ccc810113826e2dfffb6ccd39b853f4d604b979f3fdb0f16 not found: ID does not exist"
Sep 14 08:12:02.087432 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.087401    2556 scope.go:117] "RemoveContainer" containerID="bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62"
Sep 14 08:12:02.094624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.094603    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"]
Sep 14 08:12:02.098860 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.098838    2556 scope.go:117] "RemoveContainer" containerID="bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62"
Sep 14 08:12:02.099419 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:12:02.099394    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62\": container with ID starting with bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62 not found: ID does not exist" containerID="bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62"
Sep 14 08:12:02.099524 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.099430    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62"} err="failed to get container status \"bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62\": rpc error: code = NotFound desc = could not find container \"bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62\": container with ID starting with bfe812bd132aa98321bd33694b63801d558d994b78f7b7b2a7d0ab5c0aa7bb62 not found: ID does not exist"
Sep 14 08:12:02.099524 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.099456    2556 scope.go:117] "RemoveContainer" containerID="9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70"
Sep 14 08:12:02.102068 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.102046    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2a94f0ce-5cd684b7df-l88f2"]
Sep 14 08:12:02.105104 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105084    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105106    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6824\" (UniqueName: \"kubernetes.io/projected/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-kube-api-access-n6824\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105116    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105126    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd576e4-0713-463a-a06c-02f0366d36cb-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105134    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bj52\" (UniqueName: \"kubernetes.io/projected/ddd576e4-0713-463a-a06c-02f0366d36cb-kube-api-access-7bj52\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105144    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105152    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wblmc\" (UniqueName: \"kubernetes.io/projected/578a96a0-86f6-4b96-9056-a3817c689385-kube-api-access-wblmc\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.105196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.105160    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/578a96a0-86f6-4b96-9056-a3817c689385-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:12:02.107815 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.107802    2556 scope.go:117] "RemoveContainer" containerID="9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70"
Sep 14 08:12:02.108110 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:12:02.108090    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70\": container with ID starting with 9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70 not found: ID does not exist" containerID="9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70"
Sep 14 08:12:02.108170 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.108122    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70"} err="failed to get container status \"9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70\": rpc error: code = NotFound desc = could not find container \"9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70\": container with ID starting with 9a8b43508377966c27bd3f4dda3091fc14ed931a32d9ba4f2a4c9399d87afa70 not found: ID does not exist"
Sep 14 08:12:02.111914 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.111894    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"]
Sep 14 08:12:02.116280 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.116255    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2a94f0ce-7fd69f695c-hxpsl"]
Sep 14 08:12:02.126189 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.126167    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"]
Sep 14 08:12:02.134486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:02.134465    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2a94f0ce-659c854466-9fkkz"]
Sep 14 08:12:04.019481 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:04.019408    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="578a96a0-86f6-4b96-9056-a3817c689385" path="/var/lib/kubelet/pods/578a96a0-86f6-4b96-9056-a3817c689385/volumes"
Sep 14 08:12:04.019909 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:04.019885    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" path="/var/lib/kubelet/pods/5fb1fe83-dde4-48d1-ab16-e1317ce9e03c/volumes"
Sep 14 08:12:04.020243 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:04.020228    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd576e4-0713-463a-a06c-02f0366d36cb" path="/var/lib/kubelet/pods/ddd576e4-0713-463a-a06c-02f0366d36cb/volumes"
Sep 14 08:12:08.788660 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:08.788616    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:12:08.789101 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:08.788686    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:12:08.801483 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:08.801459    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:12:09.132768 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:09.132684    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:12:51.081151 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081112    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"]
Sep 14 08:12:51.081734 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081630    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578a96a0-86f6-4b96-9056-a3817c689385" containerName="payload-processing"
Sep 14 08:12:51.081734 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081647    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="578a96a0-86f6-4b96-9056-a3817c689385" containerName="payload-processing"
Sep 14 08:12:51.081734 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081665    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd576e4-0713-463a-a06c-02f0366d36cb" containerName="payload-pre-processing"
Sep 14 08:12:51.081734 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081673    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd576e4-0713-463a-a06c-02f0366d36cb" containerName="payload-pre-processing"
Sep 14 08:12:51.081734 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081696    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" containerName="maas-api"
Sep 14 08:12:51.081734 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081705    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" containerName="maas-api"
Sep 14 08:12:51.082162 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081786    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd576e4-0713-463a-a06c-02f0366d36cb" containerName="payload-pre-processing"
Sep 14 08:12:51.082162 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081794    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="578a96a0-86f6-4b96-9056-a3817c689385" containerName="payload-processing"
Sep 14 08:12:51.082162 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.081801    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1fe83-dde4-48d1-ab16-e1317ce9e03c" containerName="maas-api"
Sep 14 08:12:51.085210 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.085191    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.088173 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.088152    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"auth-test-model-fc0d6e-kserve-self-signed-certs\""
Sep 14 08:12:51.088351 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.088152    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"openshift-service-ca.crt\""
Sep 14 08:12:51.088351 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.088197    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"default-dockercfg-w8zwm\""
Sep 14 08:12:51.088351 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.088198    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"kube-root-ca.crt\""
Sep 14 08:12:51.095809 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.095779    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"]
Sep 14 08:12:51.164994 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.164928    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e638c228-8984-4172-ae67-34daa9fa0db1-tls-certs\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.165179 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.165033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-tmp-dir\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.165179 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.165056    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-model-cache\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.165179 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.165137    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-home\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.165291 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.165179    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d59r6\" (UniqueName: \"kubernetes.io/projected/e638c228-8984-4172-ae67-34daa9fa0db1-kube-api-access-d59r6\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.165291 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.165207    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-dshm\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.265750 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.265719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-model-cache\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.265898 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.265770    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-home\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.265898 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.265793    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d59r6\" (UniqueName: \"kubernetes.io/projected/e638c228-8984-4172-ae67-34daa9fa0db1-kube-api-access-d59r6\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.265898 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.265824    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-dshm\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.265898 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.265856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e638c228-8984-4172-ae67-34daa9fa0db1-tls-certs\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.266090 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.265910    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-tmp-dir\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.266166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.266143    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-model-cache\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.266215 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.266178    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-home\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.266263 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.266246    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-tmp-dir\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.267948 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.267923    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-dshm\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.268193 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.268177    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e638c228-8984-4172-ae67-34daa9fa0db1-tls-certs\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.272927 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.272904    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d59r6\" (UniqueName: \"kubernetes.io/projected/e638c228-8984-4172-ae67-34daa9fa0db1-kube-api-access-d59r6\") pod \"auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.395907 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.395809    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:12:51.669754 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:51.669723    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"]
Sep 14 08:12:51.670808 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:12:51.670783    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode638c228_8984_4172_ae67_34daa9fa0db1.slice/crio-393ca29e7cca758cf1b3406ea717b85c0874eb00f4a251c6885570072673d4b7 WatchSource:0}: Error finding container 393ca29e7cca758cf1b3406ea717b85c0874eb00f4a251c6885570072673d4b7: Status 404 returned error can't find the container with id 393ca29e7cca758cf1b3406ea717b85c0874eb00f4a251c6885570072673d4b7
Sep 14 08:12:52.363366 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:52.363330    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk" event={"ID":"e638c228-8984-4172-ae67-34daa9fa0db1","Type":"ContainerStarted","Data":"46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa"}
Sep 14 08:12:52.363786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:52.363378    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk" event={"ID":"e638c228-8984-4172-ae67-34daa9fa0db1","Type":"ContainerStarted","Data":"393ca29e7cca758cf1b3406ea717b85c0874eb00f4a251c6885570072673d4b7"}
Sep 14 08:12:52.379725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:12:52.379674    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk" podStartSLOduration=1.379657378 podStartE2EDuration="1.379657378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:12:52.377479833 +0000 UTC m=+1428.876872630" watchObservedRunningTime="2026-09-14 08:12:52.379657378 +0000 UTC m=+1428.879050155"
Sep 14 08:13:01.396987 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:01.396873    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:13:01.397389 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:01.397149    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:13:01.410177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:01.410138    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:13:01.425357 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:01.425326    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:13:03.682384 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.682352    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"]
Sep 14 08:13:03.685736 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.685718    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.688018 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.687996    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"embedding-w0-831f8e-kserve-self-signed-certs\""
Sep 14 08:13:03.693372 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.693348    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"]
Sep 14 08:13:03.781361 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.781328    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-model-cache\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.781517 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.781376    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-dshm\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.781517 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.781399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-home\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.781517 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.781465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tls-certs\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.781517 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.781508    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tmp-dir\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.781729 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.781592    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgrnf\" (UniqueName: \"kubernetes.io/projected/1e698d8e-f192-4132-b09d-0798a3c5a8bd-kube-api-access-zgrnf\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.882776 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.882741    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-model-cache\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.882776 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.882781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-dshm\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883006 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.882803    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-home\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883006 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.882844    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tls-certs\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883006 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.882877    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tmp-dir\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883006 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.882980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgrnf\" (UniqueName: \"kubernetes.io/projected/1e698d8e-f192-4132-b09d-0798a3c5a8bd-kube-api-access-zgrnf\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883526 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.883200    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-model-cache\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883526 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.883254    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-home\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.883526 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.883308    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tmp-dir\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.884943 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.884922    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-dshm\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.885070 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.885054    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-831f8e\"/\"embedding-w0-831f8e-kserve-self-signed-certs\""
Sep 14 08:13:03.890251 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.890230    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgrnf\" (UniqueName: \"kubernetes.io/projected/1e698d8e-f192-4132-b09d-0798a3c5a8bd-kube-api-access-zgrnf\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.895930 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.895907    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tls-certs\") pod \"embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") " pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:03.996529 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:03.996452    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:04.273403 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:04.273376    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"]
Sep 14 08:13:04.275406 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:13:04.275378    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e698d8e_f192_4132_b09d_0798a3c5a8bd.slice/crio-0565e1da679d758ca179e4ad73f8690fe01480153d45312bd36194f4aecdb5a2 WatchSource:0}: Error finding container 0565e1da679d758ca179e4ad73f8690fe01480153d45312bd36194f4aecdb5a2: Status 404 returned error can't find the container with id 0565e1da679d758ca179e4ad73f8690fe01480153d45312bd36194f4aecdb5a2
Sep 14 08:13:04.431245 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:04.431204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs" event={"ID":"1e698d8e-f192-4132-b09d-0798a3c5a8bd","Type":"ContainerStarted","Data":"cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97"}
Sep 14 08:13:04.431399 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:04.431257    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs" event={"ID":"1e698d8e-f192-4132-b09d-0798a3c5a8bd","Type":"ContainerStarted","Data":"0565e1da679d758ca179e4ad73f8690fe01480153d45312bd36194f4aecdb5a2"}
Sep 14 08:13:04.448486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:04.448438    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs" podStartSLOduration=1.4484232430000001 podStartE2EDuration="1.448423243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:13:04.447794363 +0000 UTC m=+1440.947187159" watchObservedRunningTime="2026-09-14 08:13:04.448423243 +0000 UTC m=+1440.947816058"
Sep 14 08:13:13.997219 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:13.997173    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:13.997605 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:13.997248    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:14.009474 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:14.009448    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:13:14.499718 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:13:14.499689    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:14:17.937701 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.937666    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"]
Sep 14 08:14:17.940869 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.940852    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:17.943079 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.943051    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b22414\""
Sep 14 08:14:17.952451 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.952424    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"]
Sep 14 08:14:17.980832 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.980802    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"]
Sep 14 08:14:17.984256 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.984225    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:17.986014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.985985    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5888666f-b50a-47e7-bec6-14a497b29138-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:17.986140 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.986078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvcj\" (UniqueName: \"kubernetes.io/projected/5888666f-b50a-47e7-bec6-14a497b29138-kube-api-access-gjvcj\") pod \"payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:17.986356 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.986338    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b22414-dockercfg-22sk2\""
Sep 14 08:14:17.993166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:17.993144    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"]
Sep 14 08:14:18.086897 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.086855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5888666f-b50a-47e7-bec6-14a497b29138-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:18.086897 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.086897    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwct\" (UniqueName: \"kubernetes.io/projected/a0cf9024-16f3-407b-a19f-650a093389b5-kube-api-access-xbwct\") pod \"payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") " pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.087143 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.086943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0cf9024-16f3-407b-a19f-650a093389b5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") " pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.087143 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.087012    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvcj\" (UniqueName: \"kubernetes.io/projected/5888666f-b50a-47e7-bec6-14a497b29138-kube-api-access-gjvcj\") pod \"payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:18.087655 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.087633    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5888666f-b50a-47e7-bec6-14a497b29138-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:18.094436 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.094409    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvcj\" (UniqueName: \"kubernetes.io/projected/5888666f-b50a-47e7-bec6-14a497b29138-kube-api-access-gjvcj\") pod \"payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:18.188159 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.188072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwct\" (UniqueName: \"kubernetes.io/projected/a0cf9024-16f3-407b-a19f-650a093389b5-kube-api-access-xbwct\") pod \"payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") " pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.188159 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.188128    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0cf9024-16f3-407b-a19f-650a093389b5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") " pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.188791 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.188769    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0cf9024-16f3-407b-a19f-650a093389b5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") " pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.195775 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.195751    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwct\" (UniqueName: \"kubernetes.io/projected/a0cf9024-16f3-407b-a19f-650a093389b5-kube-api-access-xbwct\") pod \"payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") " pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.260427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.260383    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:18.296065 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.296031    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.536391 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.536367    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"]
Sep 14 08:14:18.538002 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:14:18.537946    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5888666f_b50a_47e7_bec6_14a497b29138.slice/crio-472b237ece10de012dc5f0ec0eb1e7c13003f4967d100b3178d3a87dbbcb3e43 WatchSource:0}: Error finding container 472b237ece10de012dc5f0ec0eb1e7c13003f4967d100b3178d3a87dbbcb3e43: Status 404 returned error can't find the container with id 472b237ece10de012dc5f0ec0eb1e7c13003f4967d100b3178d3a87dbbcb3e43
Sep 14 08:14:18.590015 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.589988    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"]
Sep 14 08:14:18.591600 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:14:18.591576    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0cf9024_16f3_407b_a19f_650a093389b5.slice/crio-a8892a406ef5bdea9537996149d3f57e1c02e09dca4518cdaa73bac914cf08d6 WatchSource:0}: Error finding container a8892a406ef5bdea9537996149d3f57e1c02e09dca4518cdaa73bac914cf08d6: Status 404 returned error can't find the container with id a8892a406ef5bdea9537996149d3f57e1c02e09dca4518cdaa73bac914cf08d6
Sep 14 08:14:18.848725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.848628    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch" event={"ID":"5888666f-b50a-47e7-bec6-14a497b29138","Type":"ContainerStarted","Data":"3feca581020f62c016eebbf1e53730c69eec4acdf28fa62fce764ba6779a59e4"}
Sep 14 08:14:18.848725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.848692    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch" event={"ID":"5888666f-b50a-47e7-bec6-14a497b29138","Type":"ContainerStarted","Data":"472b237ece10de012dc5f0ec0eb1e7c13003f4967d100b3178d3a87dbbcb3e43"}
Sep 14 08:14:18.848725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.848723    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:18.850179 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.850143    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr" event={"ID":"a0cf9024-16f3-407b-a19f-650a093389b5","Type":"ContainerStarted","Data":"9fa1e7b6c63528cc8d53c5e42b5f7dd758a886f57ad9f44edb65b753aa79e14f"}
Sep 14 08:14:18.850300 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.850190    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:18.850300 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.850206    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr" event={"ID":"a0cf9024-16f3-407b-a19f-650a093389b5","Type":"ContainerStarted","Data":"a8892a406ef5bdea9537996149d3f57e1c02e09dca4518cdaa73bac914cf08d6"}
Sep 14 08:14:18.864096 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.864048    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch" podStartSLOduration=1.864034113 podStartE2EDuration="1.864034113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:18.86302359 +0000 UTC m=+1515.362416369" watchObservedRunningTime="2026-09-14 08:14:18.864034113 +0000 UTC m=+1515.363426890"
Sep 14 08:14:18.877509 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:18.877467    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr" podStartSLOduration=1.877453401 podStartE2EDuration="1.877453401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:18.876741832 +0000 UTC m=+1515.376134630" watchObservedRunningTime="2026-09-14 08:14:18.877453401 +0000 UTC m=+1515.376846180"
Sep 14 08:14:20.560911 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:20.560874    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"]
Sep 14 08:14:20.565228 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:20.565199    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"]
Sep 14 08:14:20.575451 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:20.575426    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"]
Sep 14 08:14:20.584292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:20.584268    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"]
Sep 14 08:14:29.860561 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:29.860481    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:14:29.860561 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:29.860558    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:14:31.444122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.444086    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"]
Sep 14 08:14:31.444481 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.444319    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s" podUID="c7fc9632-deb5-45ec-8d72-26aa1e42fc54" containerName="payload-processing" containerID="cri-o://9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a" gracePeriod=30
Sep 14 08:14:31.701715 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.701657    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:14:31.817167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.817123    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4m8c\" (UniqueName: \"kubernetes.io/projected/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-kube-api-access-t4m8c\") pod \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") "
Sep 14 08:14:31.817357 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.817275    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-plugins-config-volume\") pod \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\" (UID: \"c7fc9632-deb5-45ec-8d72-26aa1e42fc54\") "
Sep 14 08:14:31.817613 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.817586    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7fc9632-deb5-45ec-8d72-26aa1e42fc54" (UID: "c7fc9632-deb5-45ec-8d72-26aa1e42fc54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:14:31.819276 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.819256    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-kube-api-access-t4m8c" (OuterVolumeSpecName: "kube-api-access-t4m8c") pod "c7fc9632-deb5-45ec-8d72-26aa1e42fc54" (UID: "c7fc9632-deb5-45ec-8d72-26aa1e42fc54"). InnerVolumeSpecName "kube-api-access-t4m8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:31.918587 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.918550    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:31.918587 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.918582    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4m8c\" (UniqueName: \"kubernetes.io/projected/c7fc9632-deb5-45ec-8d72-26aa1e42fc54-kube-api-access-t4m8c\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:31.927674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.927641    2556 generic.go:356] "Generic (PLEG): container finished" podID="c7fc9632-deb5-45ec-8d72-26aa1e42fc54" containerID="9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a" exitCode=0
Sep 14 08:14:31.927826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.927687    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"
Sep 14 08:14:31.927826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.927721    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s" event={"ID":"c7fc9632-deb5-45ec-8d72-26aa1e42fc54","Type":"ContainerDied","Data":"9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a"}
Sep 14 08:14:31.927826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.927783    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s" event={"ID":"c7fc9632-deb5-45ec-8d72-26aa1e42fc54","Type":"ContainerDied","Data":"0ad3e5ae5ea9a7af2cc8bd32fa6b839645c2a1918ed0b8eefd5a320f088d52ee"}
Sep 14 08:14:31.927826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.927812    2556 scope.go:117] "RemoveContainer" containerID="9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a"
Sep 14 08:14:31.937161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.937141    2556 scope.go:117] "RemoveContainer" containerID="9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a"
Sep 14 08:14:31.937453 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:14:31.937435    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a\": container with ID starting with 9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a not found: ID does not exist" containerID="9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a"
Sep 14 08:14:31.937507 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.937461    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a"} err="failed to get container status \"9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a\": rpc error: code = NotFound desc = could not find container \"9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a\": container with ID starting with 9f428434bccb19c0cae88ecbc4912291870b5d90f03a825b4591ca507d31781a not found: ID does not exist"
Sep 14 08:14:31.949420 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.949396    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"]
Sep 14 08:14:31.952098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:31.952037    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-831f8e-5b7b9df88f-b5f5s"]
Sep 14 08:14:32.019782 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:32.019748    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7fc9632-deb5-45ec-8d72-26aa1e42fc54" path="/var/lib/kubelet/pods/c7fc9632-deb5-45ec-8d72-26aa1e42fc54/volumes"
Sep 14 08:14:35.583898 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.583853    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs" podUID="1ef42a8e-fda2-4676-b394-6c3e92755b9f" containerName="main" containerID="cri-o://1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3" gracePeriod=45
Sep 14 08:14:35.589220 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.589185    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls" podUID="7347bf8e-20ee-418f-9eb3-afec682d8517" containerName="main" containerID="cri-o://b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512" gracePeriod=45
Sep 14 08:14:35.600373 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.600342    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs" podUID="1e698d8e-f192-4132-b09d-0798a3c5a8bd" containerName="main" containerID="cri-o://cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97" gracePeriod=45
Sep 14 08:14:35.602715 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.602655    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8" podUID="14ce6879-bdf6-4b39-a734-ba5741eed7aa" containerName="main" containerID="cri-o://14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f" gracePeriod=45
Sep 14 08:14:35.852140 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.852113    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:14:35.902315 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.902281    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:14:35.925222 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.924814    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:14:35.940371 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.940170    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:14:35.963039 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.963007    2556 generic.go:356] "Generic (PLEG): container finished" podID="7347bf8e-20ee-418f-9eb3-afec682d8517" containerID="b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512" exitCode=0
Sep 14 08:14:35.963209 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.963092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls" event={"ID":"7347bf8e-20ee-418f-9eb3-afec682d8517","Type":"ContainerDied","Data":"b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512"}
Sep 14 08:14:35.963209 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.963101    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"
Sep 14 08:14:35.963209 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.963122    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls" event={"ID":"7347bf8e-20ee-418f-9eb3-afec682d8517","Type":"ContainerDied","Data":"8c49fdbab2da532ea9712f8963e29b61905ceaf8a3ae7a3a52d3a48c5054db61"}
Sep 14 08:14:35.963209 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.963142    2556 scope.go:117] "RemoveContainer" containerID="b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512"
Sep 14 08:14:35.965073 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965051    2556 generic.go:356] "Generic (PLEG): container finished" podID="1e698d8e-f192-4132-b09d-0798a3c5a8bd" containerID="cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97" exitCode=0
Sep 14 08:14:35.965190 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965176    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"
Sep 14 08:14:35.965303 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965286    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs" event={"ID":"1e698d8e-f192-4132-b09d-0798a3c5a8bd","Type":"ContainerDied","Data":"cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97"}
Sep 14 08:14:35.965367 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965318    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs" event={"ID":"1e698d8e-f192-4132-b09d-0798a3c5a8bd","Type":"ContainerDied","Data":"0565e1da679d758ca179e4ad73f8690fe01480153d45312bd36194f4aecdb5a2"}
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965813    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-dshm\") pod \"7347bf8e-20ee-418f-9eb3-afec682d8517\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965850    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tls-certs\") pod \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965894    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-tmp-dir\") pod \"7347bf8e-20ee-418f-9eb3-afec682d8517\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.965922    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjb6l\" (UniqueName: \"kubernetes.io/projected/7347bf8e-20ee-418f-9eb3-afec682d8517-kube-api-access-cjb6l\") pod \"7347bf8e-20ee-418f-9eb3-afec682d8517\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966010    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-dshm\") pod \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966057    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7347bf8e-20ee-418f-9eb3-afec682d8517-tls-certs\") pod \"7347bf8e-20ee-418f-9eb3-afec682d8517\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966102    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-home\") pod \"7347bf8e-20ee-418f-9eb3-afec682d8517\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") "
Sep 14 08:14:35.966626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966146    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dpgz\" (UniqueName: \"kubernetes.io/projected/1ef42a8e-fda2-4676-b394-6c3e92755b9f-kube-api-access-7dpgz\") pod \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") "
Sep 14 08:14:35.967161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966750    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-home" (OuterVolumeSpecName: "home") pod "7347bf8e-20ee-418f-9eb3-afec682d8517" (UID: "7347bf8e-20ee-418f-9eb3-afec682d8517"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.967161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966767    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-home\") pod \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") "
Sep 14 08:14:35.967161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966834    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tmp-dir\") pod \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") "
Sep 14 08:14:35.967161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966864    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-model-cache\") pod \"7347bf8e-20ee-418f-9eb3-afec682d8517\" (UID: \"7347bf8e-20ee-418f-9eb3-afec682d8517\") "
Sep 14 08:14:35.967161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966909    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-model-cache\") pod \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\" (UID: \"1ef42a8e-fda2-4676-b394-6c3e92755b9f\") "
Sep 14 08:14:35.967441 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.967306    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ef42a8e-fda2-4676-b394-6c3e92755b9f" (UID: "1ef42a8e-fda2-4676-b394-6c3e92755b9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.967441 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.967334    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:35.967550 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.967501    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ef42a8e-fda2-4676-b394-6c3e92755b9f" (UID: "1ef42a8e-fda2-4676-b394-6c3e92755b9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.967674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.967633    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-home" (OuterVolumeSpecName: "home") pod "1ef42a8e-fda2-4676-b394-6c3e92755b9f" (UID: "1ef42a8e-fda2-4676-b394-6c3e92755b9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.967780 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.967671    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-model-cache" (OuterVolumeSpecName: "model-cache") pod "7347bf8e-20ee-418f-9eb3-afec682d8517" (UID: "7347bf8e-20ee-418f-9eb3-afec682d8517"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.967859 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.966717    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7347bf8e-20ee-418f-9eb3-afec682d8517" (UID: "7347bf8e-20ee-418f-9eb3-afec682d8517"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.968387 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.968359    2556 generic.go:356] "Generic (PLEG): container finished" podID="1ef42a8e-fda2-4676-b394-6c3e92755b9f" containerID="1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3" exitCode=0
Sep 14 08:14:35.968549 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.968531    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"
Sep 14 08:14:35.968698 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.968674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs" event={"ID":"1ef42a8e-fda2-4676-b394-6c3e92755b9f","Type":"ContainerDied","Data":"1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3"}
Sep 14 08:14:35.968763 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.968716    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs" event={"ID":"1ef42a8e-fda2-4676-b394-6c3e92755b9f","Type":"ContainerDied","Data":"b9627e8713111afc31ef04c9356f47ee1bbff130597cecbbef385088b501d38d"}
Sep 14 08:14:35.969336 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.969307    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7347bf8e-20ee-418f-9eb3-afec682d8517-kube-api-access-cjb6l" (OuterVolumeSpecName: "kube-api-access-cjb6l") pod "7347bf8e-20ee-418f-9eb3-afec682d8517" (UID: "7347bf8e-20ee-418f-9eb3-afec682d8517"). InnerVolumeSpecName "kube-api-access-cjb6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:35.969836 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.969806    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef42a8e-fda2-4676-b394-6c3e92755b9f-kube-api-access-7dpgz" (OuterVolumeSpecName: "kube-api-access-7dpgz") pod "1ef42a8e-fda2-4676-b394-6c3e92755b9f" (UID: "1ef42a8e-fda2-4676-b394-6c3e92755b9f"). InnerVolumeSpecName "kube-api-access-7dpgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:35.970740 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.970712    2556 generic.go:356] "Generic (PLEG): container finished" podID="14ce6879-bdf6-4b39-a734-ba5741eed7aa" containerID="14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f" exitCode=0
Sep 14 08:14:35.970831 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.970757    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8" event={"ID":"14ce6879-bdf6-4b39-a734-ba5741eed7aa","Type":"ContainerDied","Data":"14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f"}
Sep 14 08:14:35.970831 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.970783    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8" event={"ID":"14ce6879-bdf6-4b39-a734-ba5741eed7aa","Type":"ContainerDied","Data":"1f8b524e05837f07a96b041179ef87d07c574aa17bbd86dcf7b23258d34a784a"}
Sep 14 08:14:35.970953 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.970841    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"
Sep 14 08:14:35.971812 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.971776    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7347bf8e-20ee-418f-9eb3-afec682d8517-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7347bf8e-20ee-418f-9eb3-afec682d8517" (UID: "7347bf8e-20ee-418f-9eb3-afec682d8517"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:35.973801 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.973767    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-dshm" (OuterVolumeSpecName: "dshm") pod "7347bf8e-20ee-418f-9eb3-afec682d8517" (UID: "7347bf8e-20ee-418f-9eb3-afec682d8517"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.974203 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.974182    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-dshm" (OuterVolumeSpecName: "dshm") pod "1ef42a8e-fda2-4676-b394-6c3e92755b9f" (UID: "1ef42a8e-fda2-4676-b394-6c3e92755b9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:35.975390 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.975366    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ef42a8e-fda2-4676-b394-6c3e92755b9f" (UID: "1ef42a8e-fda2-4676-b394-6c3e92755b9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:35.977202 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.977177    2556 scope.go:117] "RemoveContainer" containerID="b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512"
Sep 14 08:14:35.977522 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:14:35.977498    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512\": container with ID starting with b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512 not found: ID does not exist" containerID="b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512"
Sep 14 08:14:35.977635 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.977533    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512"} err="failed to get container status \"b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512\": rpc error: code = NotFound desc = could not find container \"b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512\": container with ID starting with b310fbe0587ff09bf84be97ef072c6b435f732ff79fbe26a0d38ea2e0adca512 not found: ID does not exist"
Sep 14 08:14:35.977635 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.977553    2556 scope.go:117] "RemoveContainer" containerID="cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97"
Sep 14 08:14:35.988026 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.988002    2556 scope.go:117] "RemoveContainer" containerID="cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97"
Sep 14 08:14:35.988556 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:14:35.988498    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97\": container with ID starting with cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97 not found: ID does not exist" containerID="cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97"
Sep 14 08:14:35.988658 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.988554    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97"} err="failed to get container status \"cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97\": rpc error: code = NotFound desc = could not find container \"cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97\": container with ID starting with cb3fbae02491d4eb6e77f5638ad723099a62bb0f9618edd1283a4fe6013d7e97 not found: ID does not exist"
Sep 14 08:14:35.988658 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.988580    2556 scope.go:117] "RemoveContainer" containerID="1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3"
Sep 14 08:14:35.999724 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:35.999707    2556 scope.go:117] "RemoveContainer" containerID="1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3"
Sep 14 08:14:36.000043 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:14:36.000017    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3\": container with ID starting with 1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3 not found: ID does not exist" containerID="1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3"
Sep 14 08:14:36.000129 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.000054    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3"} err="failed to get container status \"1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3\": rpc error: code = NotFound desc = could not find container \"1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3\": container with ID starting with 1a03407d9b72a96ee52912fc444ac199197dbfdbd9f780a46a9c2d71d596f9c3 not found: ID does not exist"
Sep 14 08:14:36.000129 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.000077    2556 scope.go:117] "RemoveContainer" containerID="14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f"
Sep 14 08:14:36.009309 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.009284    2556 scope.go:117] "RemoveContainer" containerID="14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f"
Sep 14 08:14:36.009571 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:14:36.009551    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f\": container with ID starting with 14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f not found: ID does not exist" containerID="14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f"
Sep 14 08:14:36.009633 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.009580    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f"} err="failed to get container status \"14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f\": rpc error: code = NotFound desc = could not find container \"14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f\": container with ID starting with 14b753ffb89a4fc83943f422edb494cb64479178999169a515efd38b4b1f216f not found: ID does not exist"
Sep 14 08:14:36.068603 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068562    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tls-certs\") pod \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") "
Sep 14 08:14:36.068736 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068619    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgrnf\" (UniqueName: \"kubernetes.io/projected/1e698d8e-f192-4132-b09d-0798a3c5a8bd-kube-api-access-zgrnf\") pod \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") "
Sep 14 08:14:36.068736 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068638    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tls-certs\") pod \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") "
Sep 14 08:14:36.068736 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068662    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9l2q\" (UniqueName: \"kubernetes.io/projected/14ce6879-bdf6-4b39-a734-ba5741eed7aa-kube-api-access-n9l2q\") pod \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") "
Sep 14 08:14:36.068924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068777    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-model-cache\") pod \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") "
Sep 14 08:14:36.068924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068827    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tmp-dir\") pod \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") "
Sep 14 08:14:36.068924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068865    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-dshm\") pod \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") "
Sep 14 08:14:36.068924 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.068893    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tmp-dir\") pod \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") "
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069046    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-home\") pod \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") "
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069079    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-dshm\") pod \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") "
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069110    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-home\") pod \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\" (UID: \"14ce6879-bdf6-4b39-a734-ba5741eed7aa\") "
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069112    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "14ce6879-bdf6-4b39-a734-ba5741eed7aa" (UID: "14ce6879-bdf6-4b39-a734-ba5741eed7aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069165    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-model-cache\") pod \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\" (UID: \"1e698d8e-f192-4132-b09d-0798a3c5a8bd\") "
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069376    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-home" (OuterVolumeSpecName: "home") pod "1e698d8e-f192-4132-b09d-0798a3c5a8bd" (UID: "1e698d8e-f192-4132-b09d-0798a3c5a8bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069482    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "14ce6879-bdf6-4b39-a734-ba5741eed7aa" (UID: "14ce6879-bdf6-4b39-a734-ba5741eed7aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069602    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "1e698d8e-f192-4132-b09d-0798a3c5a8bd" (UID: "1e698d8e-f192-4132-b09d-0798a3c5a8bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.069695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069670    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1e698d8e-f192-4132-b09d-0798a3c5a8bd" (UID: "1e698d8e-f192-4132-b09d-0798a3c5a8bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069737    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069756    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069784    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069799    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069812    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069824    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069836    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef42a8e-fda2-4676-b394-6c3e92755b9f-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069850    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7347bf8e-20ee-418f-9eb3-afec682d8517-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069853    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-home" (OuterVolumeSpecName: "home") pod "14ce6879-bdf6-4b39-a734-ba5741eed7aa" (UID: "14ce6879-bdf6-4b39-a734-ba5741eed7aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069862    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069907    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjb6l\" (UniqueName: \"kubernetes.io/projected/7347bf8e-20ee-418f-9eb3-afec682d8517-kube-api-access-cjb6l\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069927    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069944    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef42a8e-fda2-4676-b394-6c3e92755b9f-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.069990    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7347bf8e-20ee-418f-9eb3-afec682d8517-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.070003    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.070168 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.070016    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dpgz\" (UniqueName: \"kubernetes.io/projected/1ef42a8e-fda2-4676-b394-6c3e92755b9f-kube-api-access-7dpgz\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.071353 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.071329    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "14ce6879-bdf6-4b39-a734-ba5741eed7aa" (UID: "14ce6879-bdf6-4b39-a734-ba5741eed7aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:36.071445 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.071345    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1e698d8e-f192-4132-b09d-0798a3c5a8bd" (UID: "1e698d8e-f192-4132-b09d-0798a3c5a8bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:14:36.071445 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.071400    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-dshm" (OuterVolumeSpecName: "dshm") pod "14ce6879-bdf6-4b39-a734-ba5741eed7aa" (UID: "14ce6879-bdf6-4b39-a734-ba5741eed7aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.071551 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.071442    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e698d8e-f192-4132-b09d-0798a3c5a8bd-kube-api-access-zgrnf" (OuterVolumeSpecName: "kube-api-access-zgrnf") pod "1e698d8e-f192-4132-b09d-0798a3c5a8bd" (UID: "1e698d8e-f192-4132-b09d-0798a3c5a8bd"). InnerVolumeSpecName "kube-api-access-zgrnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:36.071906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.071884    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-dshm" (OuterVolumeSpecName: "dshm") pod "1e698d8e-f192-4132-b09d-0798a3c5a8bd" (UID: "1e698d8e-f192-4132-b09d-0798a3c5a8bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:14:36.071906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.071899    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14ce6879-bdf6-4b39-a734-ba5741eed7aa-kube-api-access-n9l2q" (OuterVolumeSpecName: "kube-api-access-n9l2q") pod "14ce6879-bdf6-4b39-a734-ba5741eed7aa" (UID: "14ce6879-bdf6-4b39-a734-ba5741eed7aa"). InnerVolumeSpecName "kube-api-access-n9l2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:36.171275 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171225    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171275 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171258    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgrnf\" (UniqueName: \"kubernetes.io/projected/1e698d8e-f192-4132-b09d-0798a3c5a8bd-kube-api-access-zgrnf\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171275 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171273    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ce6879-bdf6-4b39-a734-ba5741eed7aa-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171290    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9l2q\" (UniqueName: \"kubernetes.io/projected/14ce6879-bdf6-4b39-a734-ba5741eed7aa-kube-api-access-n9l2q\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171303    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171317    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171329    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e698d8e-f192-4132-b09d-0798a3c5a8bd-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.171562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.171340    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ce6879-bdf6-4b39-a734-ba5741eed7aa-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:14:36.279890 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.279820    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"]
Sep 14 08:14:36.282698 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.282672    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-w0-831f8e-kserve-788546bbb-pq6ls"]
Sep 14 08:14:36.290473 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.290448    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"]
Sep 14 08:14:36.298785 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.297121    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/embedding-w0-831f8e-kserve-7ddf85cd67-6qvqs"]
Sep 14 08:14:36.308647 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.308621    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"]
Sep 14 08:14:36.312041 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.312014    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/distinct-2-w0-831f8e-kserve-64d4744589-z6jhs"]
Sep 14 08:14:36.322402 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.322378    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"]
Sep 14 08:14:36.325856 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:36.325835    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-831f8e/premium-w0-831f8e-kserve-578575d967-7nnr8"]
Sep 14 08:14:38.018923 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:38.018889    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14ce6879-bdf6-4b39-a734-ba5741eed7aa" path="/var/lib/kubelet/pods/14ce6879-bdf6-4b39-a734-ba5741eed7aa/volumes"
Sep 14 08:14:38.019367 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:38.019348    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e698d8e-f192-4132-b09d-0798a3c5a8bd" path="/var/lib/kubelet/pods/1e698d8e-f192-4132-b09d-0798a3c5a8bd/volumes"
Sep 14 08:14:38.019697 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:38.019685    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef42a8e-fda2-4676-b394-6c3e92755b9f" path="/var/lib/kubelet/pods/1ef42a8e-fda2-4676-b394-6c3e92755b9f/volumes"
Sep 14 08:14:38.020351 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:38.020335    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7347bf8e-20ee-418f-9eb3-afec682d8517" path="/var/lib/kubelet/pods/7347bf8e-20ee-418f-9eb3-afec682d8517/volumes"
Sep 14 08:14:46.629422 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629386    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"]
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629792    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14ce6879-bdf6-4b39-a734-ba5741eed7aa" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629807    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="14ce6879-bdf6-4b39-a734-ba5741eed7aa" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629815    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef42a8e-fda2-4676-b394-6c3e92755b9f" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629820    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef42a8e-fda2-4676-b394-6c3e92755b9f" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629830    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e698d8e-f192-4132-b09d-0798a3c5a8bd" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629836    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e698d8e-f192-4132-b09d-0798a3c5a8bd" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629851    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7fc9632-deb5-45ec-8d72-26aa1e42fc54" containerName="payload-processing"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629856    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7fc9632-deb5-45ec-8d72-26aa1e42fc54" containerName="payload-processing"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629865    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7347bf8e-20ee-418f-9eb3-afec682d8517" containerName="main"
Sep 14 08:14:46.629872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629869    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7347bf8e-20ee-418f-9eb3-afec682d8517" containerName="main"
Sep 14 08:14:46.630207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629929    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e698d8e-f192-4132-b09d-0798a3c5a8bd" containerName="main"
Sep 14 08:14:46.630207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629936    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7fc9632-deb5-45ec-8d72-26aa1e42fc54" containerName="payload-processing"
Sep 14 08:14:46.630207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629943    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7347bf8e-20ee-418f-9eb3-afec682d8517" containerName="main"
Sep 14 08:14:46.630207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629954    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef42a8e-fda2-4676-b394-6c3e92755b9f" containerName="main"
Sep 14 08:14:46.630207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.629976    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="14ce6879-bdf6-4b39-a734-ba5741eed7aa" containerName="main"
Sep 14 08:14:46.635167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.635137    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.637265 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.637244    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8e721c-serving-cert\""
Sep 14 08:14:46.645108 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.645081    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"]
Sep 14 08:14:46.767488 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.767449    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xmpd\" (UniqueName: \"kubernetes.io/projected/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-kube-api-access-2xmpd\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.767651 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.767499    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-metrics-certs\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.767651 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.767558    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.767651 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.767576    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-maas-api-tls\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.868407 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.868372    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.868407 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.868408    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-maas-api-tls\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.868638 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.868495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmpd\" (UniqueName: \"kubernetes.io/projected/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-kube-api-access-2xmpd\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.868638 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.868548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-metrics-certs\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.869211 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.869179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.870876 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.870852    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-metrics-certs\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.871001 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.870876    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-maas-api-tls\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.876208 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.876186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmpd\" (UniqueName: \"kubernetes.io/projected/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-kube-api-access-2xmpd\") pod \"maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:46.947301 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:46.947276    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:47.214020 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.213936    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"]
Sep 14 08:14:47.218625 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.218608    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.221188 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.221163    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-8e721c\""
Sep 14 08:14:47.228458 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.228435    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"]
Sep 14 08:14:47.233140 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.233116    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"]
Sep 14 08:14:47.233953 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:14:47.233930    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c2e61e_66dd_4fb8_92d6_ef60c3f418e5.slice/crio-1cd9b1a868d46878f3d370f3b94517961313b71359652964be2f8bb02878ab13 WatchSource:0}: Error finding container 1cd9b1a868d46878f3d370f3b94517961313b71359652964be2f8bb02878ab13: Status 404 returned error can't find the container with id 1cd9b1a868d46878f3d370f3b94517961313b71359652964be2f8bb02878ab13
Sep 14 08:14:47.235814 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.235795    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:14:47.267884 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.267852    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"]
Sep 14 08:14:47.271872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.271855    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.274098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.274077    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-8e721c-dockercfg-ktx7z\""
Sep 14 08:14:47.281237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.281216    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"]
Sep 14 08:14:47.372881 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.372837    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7jjd\" (UniqueName: \"kubernetes.io/projected/261c76ab-f598-4b3f-8ab7-8e11a4445510-kube-api-access-m7jjd\") pod \"payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.373048 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.372945    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk2qs\" (UniqueName: \"kubernetes.io/projected/19753b29-4cfc-45da-b91a-73fd693b1d25-kube-api-access-pk2qs\") pod \"payload-processing-e2e-api-b-8e721c-84f8644765-t8744\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.373118 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.373056    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19753b29-4cfc-45da-b91a-73fd693b1d25-plugins-config-volume\") pod \"payload-processing-e2e-api-b-8e721c-84f8644765-t8744\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.373118 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.373076    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c76ab-f598-4b3f-8ab7-8e11a4445510-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.474309 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.474209    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19753b29-4cfc-45da-b91a-73fd693b1d25-plugins-config-volume\") pod \"payload-processing-e2e-api-b-8e721c-84f8644765-t8744\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.474309 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.474267    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c76ab-f598-4b3f-8ab7-8e11a4445510-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.474539 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.474343    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7jjd\" (UniqueName: \"kubernetes.io/projected/261c76ab-f598-4b3f-8ab7-8e11a4445510-kube-api-access-m7jjd\") pod \"payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.474539 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.474398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk2qs\" (UniqueName: \"kubernetes.io/projected/19753b29-4cfc-45da-b91a-73fd693b1d25-kube-api-access-pk2qs\") pod \"payload-processing-e2e-api-b-8e721c-84f8644765-t8744\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.474999 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.474978    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19753b29-4cfc-45da-b91a-73fd693b1d25-plugins-config-volume\") pod \"payload-processing-e2e-api-b-8e721c-84f8644765-t8744\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.475181 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.475158    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c76ab-f598-4b3f-8ab7-8e11a4445510-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.482140 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.482116    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk2qs\" (UniqueName: \"kubernetes.io/projected/19753b29-4cfc-45da-b91a-73fd693b1d25-kube-api-access-pk2qs\") pod \"payload-processing-e2e-api-b-8e721c-84f8644765-t8744\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.482245 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.482187    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7jjd\" (UniqueName: \"kubernetes.io/projected/261c76ab-f598-4b3f-8ab7-8e11a4445510-kube-api-access-m7jjd\") pod \"payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.531358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.531326    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:47.583776 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.583734    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:47.804246 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.804214    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"]
Sep 14 08:14:47.805451 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:14:47.805420    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261c76ab_f598_4b3f_8ab7_8e11a4445510.slice/crio-862d3cef9c4ef5172a9e671a08eae14d04e3f03635a6a007ce74e560649c8563 WatchSource:0}: Error finding container 862d3cef9c4ef5172a9e671a08eae14d04e3f03635a6a007ce74e560649c8563: Status 404 returned error can't find the container with id 862d3cef9c4ef5172a9e671a08eae14d04e3f03635a6a007ce74e560649c8563
Sep 14 08:14:47.881102 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:47.881074    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"]
Sep 14 08:14:47.882803 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:14:47.882776    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19753b29_4cfc_45da_b91a_73fd693b1d25.slice/crio-41ce481a93555d1d0c42c475324cca663e36187adde46e31478813b981dcdbe9 WatchSource:0}: Error finding container 41ce481a93555d1d0c42c475324cca663e36187adde46e31478813b981dcdbe9: Status 404 returned error can't find the container with id 41ce481a93555d1d0c42c475324cca663e36187adde46e31478813b981dcdbe9
Sep 14 08:14:48.046053 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.046018    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" event={"ID":"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5","Type":"ContainerStarted","Data":"efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a"}
Sep 14 08:14:48.046234 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.046078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" event={"ID":"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5","Type":"ContainerStarted","Data":"1cd9b1a868d46878f3d370f3b94517961313b71359652964be2f8bb02878ab13"}
Sep 14 08:14:48.046234 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.046178    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:48.047902 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.047871    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk" event={"ID":"261c76ab-f598-4b3f-8ab7-8e11a4445510","Type":"ContainerStarted","Data":"ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e"}
Sep 14 08:14:48.048056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.047921    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk" event={"ID":"261c76ab-f598-4b3f-8ab7-8e11a4445510","Type":"ContainerStarted","Data":"862d3cef9c4ef5172a9e671a08eae14d04e3f03635a6a007ce74e560649c8563"}
Sep 14 08:14:48.048056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.047951    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:48.049901 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.049871    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744" event={"ID":"19753b29-4cfc-45da-b91a-73fd693b1d25","Type":"ContainerStarted","Data":"a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6"}
Sep 14 08:14:48.050044 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.049918    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:14:48.050044 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.049936    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744" event={"ID":"19753b29-4cfc-45da-b91a-73fd693b1d25","Type":"ContainerStarted","Data":"41ce481a93555d1d0c42c475324cca663e36187adde46e31478813b981dcdbe9"}
Sep 14 08:14:48.062191 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.062094    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" podStartSLOduration=1.657741325 podStartE2EDuration="2.06208078s" totalImagesPullingTime="404.339455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:48.061572606 +0000 UTC m=+1544.560965415" watchObservedRunningTime="2026-09-14 08:14:48.06208078 +0000 UTC m=+1544.561473557"
Sep 14 08:14:48.078947 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.078904    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk" podStartSLOduration=1.07888821 podStartE2EDuration="1.07888821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:48.075924806 +0000 UTC m=+1544.575317606" watchObservedRunningTime="2026-09-14 08:14:48.07888821 +0000 UTC m=+1544.578280987"
Sep 14 08:14:48.092021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:48.091981    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744" podStartSLOduration=1.091954177 podStartE2EDuration="1.091954177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:48.089597891 +0000 UTC m=+1544.588990682" watchObservedRunningTime="2026-09-14 08:14:48.091954177 +0000 UTC m=+1544.591346954"
Sep 14 08:14:54.063107 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:54.063073    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:14:59.059204 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:59.059157    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:14:59.059676 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:14:59.059345    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:15:00.143622 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.143577    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"]
Sep 14 08:15:00.148352 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.148332    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"
Sep 14 08:15:00.150615 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.150594    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:15:00.155483 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.155372    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"]
Sep 14 08:15:00.160292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.160271    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"
Sep 14 08:15:00.166284 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.166261    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"]
Sep 14 08:15:00.169946 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.169927    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"
Sep 14 08:15:00.180284 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.180208    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"]
Sep 14 08:15:00.200665 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.200626    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"
Sep 14 08:15:00.440857 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.440834    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"]
Sep 14 08:15:00.442660 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:15:00.442632    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25282b06_1c47_4813_ade6_5c75003d69fa.slice/crio-7eae7db8b29b54ca5544d6e6cb57dd27aa227b3cf5e55fe80c9d21d8b3f105cd WatchSource:0}: Error finding container 7eae7db8b29b54ca5544d6e6cb57dd27aa227b3cf5e55fe80c9d21d8b3f105cd: Status 404 returned error can't find the container with id 7eae7db8b29b54ca5544d6e6cb57dd27aa227b3cf5e55fe80c9d21d8b3f105cd
Sep 14 08:15:00.481989 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:00.481950    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"]
Sep 14 08:15:00.483085 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:15:00.483054    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06878ac5_98fb_4b1c_8486_a8486ec786d3.slice/crio-f833d30dfcdc8fbac7262bb43a5c0a0a19c78a544b5224c421ca25965302b58d WatchSource:0}: Error finding container f833d30dfcdc8fbac7262bb43a5c0a0a19c78a544b5224c421ca25965302b58d: Status 404 returned error can't find the container with id f833d30dfcdc8fbac7262bb43a5c0a0a19c78a544b5224c421ca25965302b58d
Sep 14 08:15:01.125184 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:01.125147    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c" event={"ID":"25282b06-1c47-4813-ade6-5c75003d69fa","Type":"ContainerStarted","Data":"7eae7db8b29b54ca5544d6e6cb57dd27aa227b3cf5e55fe80c9d21d8b3f105cd"}
Sep 14 08:15:01.126333 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:01.126300    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv" event={"ID":"06878ac5-98fb-4b1c-8486-a8486ec786d3","Type":"ContainerStarted","Data":"f833d30dfcdc8fbac7262bb43a5c0a0a19c78a544b5224c421ca25965302b58d"}
Sep 14 08:15:04.146720 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:04.146623    2556 generic.go:356] "Generic (PLEG): container finished" podID="06878ac5-98fb-4b1c-8486-a8486ec786d3" containerID="3345e4fa493f077402ef2bb0f31c81ddaf78ad30dc62cee828a2df2b501fbe96" exitCode=0
Sep 14 08:15:04.146720 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:04.146689    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv" event={"ID":"06878ac5-98fb-4b1c-8486-a8486ec786d3","Type":"ContainerDied","Data":"3345e4fa493f077402ef2bb0f31c81ddaf78ad30dc62cee828a2df2b501fbe96"}
Sep 14 08:15:04.148323 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:04.148302    2556 generic.go:356] "Generic (PLEG): container finished" podID="25282b06-1c47-4813-ade6-5c75003d69fa" containerID="4a69f91567772de2efaa6d306db0837ee01b7cee0aa81bfe6506dd96b5386022" exitCode=0
Sep 14 08:15:04.148419 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:04.148380    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c" event={"ID":"25282b06-1c47-4813-ade6-5c75003d69fa","Type":"ContainerDied","Data":"4a69f91567772de2efaa6d306db0837ee01b7cee0aa81bfe6506dd96b5386022"}
Sep 14 08:15:05.285472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:05.285445    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"
Sep 14 08:15:05.316772 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:05.316747    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"
Sep 14 08:15:06.161635 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:06.161605    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"
Sep 14 08:15:06.161816 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:06.161652    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c" event={"ID":"25282b06-1c47-4813-ade6-5c75003d69fa","Type":"ContainerDied","Data":"7eae7db8b29b54ca5544d6e6cb57dd27aa227b3cf5e55fe80c9d21d8b3f105cd"}
Sep 14 08:15:06.161816 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:06.161701    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eae7db8b29b54ca5544d6e6cb57dd27aa227b3cf5e55fe80c9d21d8b3f105cd"
Sep 14 08:15:06.163054 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:06.163028    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv" event={"ID":"06878ac5-98fb-4b1c-8486-a8486ec786d3","Type":"ContainerDied","Data":"f833d30dfcdc8fbac7262bb43a5c0a0a19c78a544b5224c421ca25965302b58d"}
Sep 14 08:15:06.163054 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:06.163048    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"
Sep 14 08:15:06.163217 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:06.163077    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f833d30dfcdc8fbac7262bb43a5c0a0a19c78a544b5224c421ca25965302b58d"
Sep 14 08:15:31.014834 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.014796    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"]
Sep 14 08:15:31.015397 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.015071    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr" podUID="a0cf9024-16f3-407b-a19f-650a093389b5" containerName="payload-processing" containerID="cri-o://9fa1e7b6c63528cc8d53c5e42b5f7dd758a886f57ad9f44edb65b753aa79e14f" gracePeriod=30
Sep 14 08:15:31.022842 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.022814    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"]
Sep 14 08:15:31.023196 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.023142    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch" podUID="5888666f-b50a-47e7-bec6-14a497b29138" containerName="payload-pre-processing" containerID="cri-o://3feca581020f62c016eebbf1e53730c69eec4acdf28fa62fce764ba6779a59e4" gracePeriod=30
Sep 14 08:15:31.303647 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.303615    2556 generic.go:356] "Generic (PLEG): container finished" podID="a0cf9024-16f3-407b-a19f-650a093389b5" containerID="9fa1e7b6c63528cc8d53c5e42b5f7dd758a886f57ad9f44edb65b753aa79e14f" exitCode=0
Sep 14 08:15:31.303755 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.303687    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr" event={"ID":"a0cf9024-16f3-407b-a19f-650a093389b5","Type":"ContainerDied","Data":"9fa1e7b6c63528cc8d53c5e42b5f7dd758a886f57ad9f44edb65b753aa79e14f"}
Sep 14 08:15:31.303802 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.303757    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr" event={"ID":"a0cf9024-16f3-407b-a19f-650a093389b5","Type":"ContainerDied","Data":"a8892a406ef5bdea9537996149d3f57e1c02e09dca4518cdaa73bac914cf08d6"}
Sep 14 08:15:31.303802 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.303784    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8892a406ef5bdea9537996149d3f57e1c02e09dca4518cdaa73bac914cf08d6"
Sep 14 08:15:31.305390 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.305367    2556 generic.go:356] "Generic (PLEG): container finished" podID="5888666f-b50a-47e7-bec6-14a497b29138" containerID="3feca581020f62c016eebbf1e53730c69eec4acdf28fa62fce764ba6779a59e4" exitCode=0
Sep 14 08:15:31.305493 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.305409    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch" event={"ID":"5888666f-b50a-47e7-bec6-14a497b29138","Type":"ContainerDied","Data":"3feca581020f62c016eebbf1e53730c69eec4acdf28fa62fce764ba6779a59e4"}
Sep 14 08:15:31.318526 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.318500    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:15:31.323486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.323465    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:15:31.402221 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402189    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjvcj\" (UniqueName: \"kubernetes.io/projected/5888666f-b50a-47e7-bec6-14a497b29138-kube-api-access-gjvcj\") pod \"5888666f-b50a-47e7-bec6-14a497b29138\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") "
Sep 14 08:15:31.402221 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402227    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbwct\" (UniqueName: \"kubernetes.io/projected/a0cf9024-16f3-407b-a19f-650a093389b5-kube-api-access-xbwct\") pod \"a0cf9024-16f3-407b-a19f-650a093389b5\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") "
Sep 14 08:15:31.402485 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402244    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0cf9024-16f3-407b-a19f-650a093389b5-plugins-config-volume\") pod \"a0cf9024-16f3-407b-a19f-650a093389b5\" (UID: \"a0cf9024-16f3-407b-a19f-650a093389b5\") "
Sep 14 08:15:31.402485 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402469    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5888666f-b50a-47e7-bec6-14a497b29138-plugins-config-volume\") pod \"5888666f-b50a-47e7-bec6-14a497b29138\" (UID: \"5888666f-b50a-47e7-bec6-14a497b29138\") "
Sep 14 08:15:31.402709 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402678    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0cf9024-16f3-407b-a19f-650a093389b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0cf9024-16f3-407b-a19f-650a093389b5" (UID: "a0cf9024-16f3-407b-a19f-650a093389b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:31.402896 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402871    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5888666f-b50a-47e7-bec6-14a497b29138-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5888666f-b50a-47e7-bec6-14a497b29138" (UID: "5888666f-b50a-47e7-bec6-14a497b29138"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:31.402996 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.402884    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0cf9024-16f3-407b-a19f-650a093389b5-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.404470 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.404439    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cf9024-16f3-407b-a19f-650a093389b5-kube-api-access-xbwct" (OuterVolumeSpecName: "kube-api-access-xbwct") pod "a0cf9024-16f3-407b-a19f-650a093389b5" (UID: "a0cf9024-16f3-407b-a19f-650a093389b5"). InnerVolumeSpecName "kube-api-access-xbwct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:31.404580 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.404501    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5888666f-b50a-47e7-bec6-14a497b29138-kube-api-access-gjvcj" (OuterVolumeSpecName: "kube-api-access-gjvcj") pod "5888666f-b50a-47e7-bec6-14a497b29138" (UID: "5888666f-b50a-47e7-bec6-14a497b29138"). InnerVolumeSpecName "kube-api-access-gjvcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:31.504136 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.504098    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjvcj\" (UniqueName: \"kubernetes.io/projected/5888666f-b50a-47e7-bec6-14a497b29138-kube-api-access-gjvcj\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.504136 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.504127    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbwct\" (UniqueName: \"kubernetes.io/projected/a0cf9024-16f3-407b-a19f-650a093389b5-kube-api-access-xbwct\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:31.504136 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:31.504137    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5888666f-b50a-47e7-bec6-14a497b29138-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:32.311741 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.311662    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"
Sep 14 08:15:32.312171 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.311669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch" event={"ID":"5888666f-b50a-47e7-bec6-14a497b29138","Type":"ContainerDied","Data":"472b237ece10de012dc5f0ec0eb1e7c13003f4967d100b3178d3a87dbbcb3e43"}
Sep 14 08:15:32.312171 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.311806    2556 scope.go:117] "RemoveContainer" containerID="3feca581020f62c016eebbf1e53730c69eec4acdf28fa62fce764ba6779a59e4"
Sep 14 08:15:32.312171 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.311934    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"
Sep 14 08:15:32.331048 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.331018    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"]
Sep 14 08:15:32.337242 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.337222    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b22414-558bb47454-44xch"]
Sep 14 08:15:32.345753 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.345734    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"]
Sep 14 08:15:32.349524 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:32.349499    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b22414-5857df9f7-fd7hr"]
Sep 14 08:15:34.019422 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:34.019386    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5888666f-b50a-47e7-bec6-14a497b29138" path="/var/lib/kubelet/pods/5888666f-b50a-47e7-bec6-14a497b29138/volumes"
Sep 14 08:15:34.019806 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:34.019754    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0cf9024-16f3-407b-a19f-650a093389b5" path="/var/lib/kubelet/pods/a0cf9024-16f3-407b-a19f-650a093389b5/volumes"
Sep 14 08:15:53.831013 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:53.830978    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"]
Sep 14 08:15:53.831439 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:53.831246    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" podUID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" containerName="maas-api" containerID="cri-o://efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a" gracePeriod=30
Sep 14 08:15:53.882132 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:53.882099    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"]
Sep 14 08:15:53.882380 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:53.882347    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744" podUID="19753b29-4cfc-45da-b91a-73fd693b1d25" containerName="payload-processing" containerID="cri-o://a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6" gracePeriod=30
Sep 14 08:15:53.892212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:53.892182    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"]
Sep 14 08:15:53.892623 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:53.892593    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk" podUID="261c76ab-f598-4b3f-8ab7-8e11a4445510" containerName="payload-pre-processing" containerID="cri-o://ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e" gracePeriod=30
Sep 14 08:15:54.148143 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.148111    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:15:54.165228 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.165197    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:15:54.196758 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.196727    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:15:54.221632 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.221596    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xmpd\" (UniqueName: \"kubernetes.io/projected/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-kube-api-access-2xmpd\") pod \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") "
Sep 14 08:15:54.221785 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.221714    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-maas-api-tls\") pod \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") "
Sep 14 08:15:54.221837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.221799    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-openshift-service-ca\") pod \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") "
Sep 14 08:15:54.221837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.221827    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-metrics-certs\") pod \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\" (UID: \"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5\") "
Sep 14 08:15:54.222239 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.222206    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" (UID: "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:54.223767 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.223735    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-kube-api-access-2xmpd" (OuterVolumeSpecName: "kube-api-access-2xmpd") pod "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" (UID: "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5"). InnerVolumeSpecName "kube-api-access-2xmpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:54.224206 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.224184    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" (UID: "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:15:54.224290 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.224211    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" (UID: "40c2e61e-66dd-4fb8-92d6-ef60c3f418e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:15:54.322740 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.322701    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19753b29-4cfc-45da-b91a-73fd693b1d25-plugins-config-volume\") pod \"19753b29-4cfc-45da-b91a-73fd693b1d25\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") "
Sep 14 08:15:54.322935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.322755    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk2qs\" (UniqueName: \"kubernetes.io/projected/19753b29-4cfc-45da-b91a-73fd693b1d25-kube-api-access-pk2qs\") pod \"19753b29-4cfc-45da-b91a-73fd693b1d25\" (UID: \"19753b29-4cfc-45da-b91a-73fd693b1d25\") "
Sep 14 08:15:54.322935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.322808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c76ab-f598-4b3f-8ab7-8e11a4445510-plugins-config-volume\") pod \"261c76ab-f598-4b3f-8ab7-8e11a4445510\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") "
Sep 14 08:15:54.322935 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.322838    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7jjd\" (UniqueName: \"kubernetes.io/projected/261c76ab-f598-4b3f-8ab7-8e11a4445510-kube-api-access-m7jjd\") pod \"261c76ab-f598-4b3f-8ab7-8e11a4445510\" (UID: \"261c76ab-f598-4b3f-8ab7-8e11a4445510\") "
Sep 14 08:15:54.323153 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.323071    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.323153 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.323084    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.323153 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.323094    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.323153 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.323104    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xmpd\" (UniqueName: \"kubernetes.io/projected/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5-kube-api-access-2xmpd\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.323153 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.323108    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19753b29-4cfc-45da-b91a-73fd693b1d25-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19753b29-4cfc-45da-b91a-73fd693b1d25" (UID: "19753b29-4cfc-45da-b91a-73fd693b1d25"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:54.323341 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.323163    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261c76ab-f598-4b3f-8ab7-8e11a4445510-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "261c76ab-f598-4b3f-8ab7-8e11a4445510" (UID: "261c76ab-f598-4b3f-8ab7-8e11a4445510"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:54.324954 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.324918    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261c76ab-f598-4b3f-8ab7-8e11a4445510-kube-api-access-m7jjd" (OuterVolumeSpecName: "kube-api-access-m7jjd") pod "261c76ab-f598-4b3f-8ab7-8e11a4445510" (UID: "261c76ab-f598-4b3f-8ab7-8e11a4445510"). InnerVolumeSpecName "kube-api-access-m7jjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:54.324954 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.324932    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19753b29-4cfc-45da-b91a-73fd693b1d25-kube-api-access-pk2qs" (OuterVolumeSpecName: "kube-api-access-pk2qs") pod "19753b29-4cfc-45da-b91a-73fd693b1d25" (UID: "19753b29-4cfc-45da-b91a-73fd693b1d25"). InnerVolumeSpecName "kube-api-access-pk2qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:54.424399 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.424373    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk2qs\" (UniqueName: \"kubernetes.io/projected/19753b29-4cfc-45da-b91a-73fd693b1d25-kube-api-access-pk2qs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.424548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.424401    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c76ab-f598-4b3f-8ab7-8e11a4445510-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.424548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.424415    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7jjd\" (UniqueName: \"kubernetes.io/projected/261c76ab-f598-4b3f-8ab7-8e11a4445510-kube-api-access-m7jjd\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.424548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.424425    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19753b29-4cfc-45da-b91a-73fd693b1d25-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:54.438165 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.438134    2556 generic.go:356] "Generic (PLEG): container finished" podID="19753b29-4cfc-45da-b91a-73fd693b1d25" containerID="a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6" exitCode=0
Sep 14 08:15:54.438311 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.438181    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"
Sep 14 08:15:54.438311 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.438223    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744" event={"ID":"19753b29-4cfc-45da-b91a-73fd693b1d25","Type":"ContainerDied","Data":"a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6"}
Sep 14 08:15:54.438311 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.438281    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744" event={"ID":"19753b29-4cfc-45da-b91a-73fd693b1d25","Type":"ContainerDied","Data":"41ce481a93555d1d0c42c475324cca663e36187adde46e31478813b981dcdbe9"}
Sep 14 08:15:54.438311 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.438306    2556 scope.go:117] "RemoveContainer" containerID="a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6"
Sep 14 08:15:54.440240 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.440187    2556 generic.go:356] "Generic (PLEG): container finished" podID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" containerID="efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a" exitCode=0
Sep 14 08:15:54.440373 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.440344    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" event={"ID":"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5","Type":"ContainerDied","Data":"efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a"}
Sep 14 08:15:54.440453 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.440367    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"
Sep 14 08:15:54.441386 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.440441    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" event={"ID":"40c2e61e-66dd-4fb8-92d6-ef60c3f418e5","Type":"ContainerDied","Data":"1cd9b1a868d46878f3d370f3b94517961313b71359652964be2f8bb02878ab13"}
Sep 14 08:15:54.446911 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.446882    2556 generic.go:356] "Generic (PLEG): container finished" podID="261c76ab-f598-4b3f-8ab7-8e11a4445510" containerID="ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e" exitCode=0
Sep 14 08:15:54.447056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.446947    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"
Sep 14 08:15:54.447056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.446988    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk" event={"ID":"261c76ab-f598-4b3f-8ab7-8e11a4445510","Type":"ContainerDied","Data":"ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e"}
Sep 14 08:15:54.447236 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.447048    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk" event={"ID":"261c76ab-f598-4b3f-8ab7-8e11a4445510","Type":"ContainerDied","Data":"862d3cef9c4ef5172a9e671a08eae14d04e3f03635a6a007ce74e560649c8563"}
Sep 14 08:15:54.455368 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.455332    2556 scope.go:117] "RemoveContainer" containerID="a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6"
Sep 14 08:15:54.455753 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:15:54.455729    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6\": container with ID starting with a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6 not found: ID does not exist" containerID="a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6"
Sep 14 08:15:54.455853 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.455762    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6"} err="failed to get container status \"a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6\": rpc error: code = NotFound desc = could not find container \"a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6\": container with ID starting with a72cdb615f50206d2cd10bdbf633fae35e411b0fcfe603c947c46c19c5e9fbc6 not found: ID does not exist"
Sep 14 08:15:54.455853 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.455789    2556 scope.go:117] "RemoveContainer" containerID="efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a"
Sep 14 08:15:54.466880 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.466859    2556 scope.go:117] "RemoveContainer" containerID="efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a"
Sep 14 08:15:54.467477 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:15:54.467451    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a\": container with ID starting with efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a not found: ID does not exist" containerID="efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a"
Sep 14 08:15:54.467580 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.467486    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a"} err="failed to get container status \"efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a\": rpc error: code = NotFound desc = could not find container \"efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a\": container with ID starting with efb6358bc212adf256b4cf08fe6fb4d6d2f69bb8758091b87fabb2a11133264a not found: ID does not exist"
Sep 14 08:15:54.467580 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.467510    2556 scope.go:117] "RemoveContainer" containerID="ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e"
Sep 14 08:15:54.469016 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.468992    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"]
Sep 14 08:15:54.471481 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.471459    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e721c-84f8644765-t8744"]
Sep 14 08:15:54.477472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.477450    2556 scope.go:117] "RemoveContainer" containerID="ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e"
Sep 14 08:15:54.477780 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:15:54.477753    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e\": container with ID starting with ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e not found: ID does not exist" containerID="ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e"
Sep 14 08:15:54.477860 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.477791    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e"} err="failed to get container status \"ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e\": rpc error: code = NotFound desc = could not find container \"ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e\": container with ID starting with ffc9775c94e80206afed7e2c8be99cfe0a7f0f64778b65bbe08e593f437ecb6e not found: ID does not exist"
Sep 14 08:15:54.481708 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.481683    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"]
Sep 14 08:15:54.484304 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.484285    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8e721c-7587d7d794-g7gtk"]
Sep 14 08:15:54.495654 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.495631    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"]
Sep 14 08:15:54.498658 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:54.498638    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c"]
Sep 14 08:15:55.350055 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350012    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"]
Sep 14 08:15:55.350429 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350421    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0cf9024-16f3-407b-a19f-650a093389b5" containerName="payload-processing"
Sep 14 08:15:55.350472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350434    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0cf9024-16f3-407b-a19f-650a093389b5" containerName="payload-processing"
Sep 14 08:15:55.350472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350451    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5888666f-b50a-47e7-bec6-14a497b29138" containerName="payload-pre-processing"
Sep 14 08:15:55.350472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350456    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5888666f-b50a-47e7-bec6-14a497b29138" containerName="payload-pre-processing"
Sep 14 08:15:55.350472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350466    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19753b29-4cfc-45da-b91a-73fd693b1d25" containerName="payload-processing"
Sep 14 08:15:55.350472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350471    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="19753b29-4cfc-45da-b91a-73fd693b1d25" containerName="payload-processing"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350483    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25282b06-1c47-4813-ade6-5c75003d69fa" containerName="cleanup"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350489    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="25282b06-1c47-4813-ade6-5c75003d69fa" containerName="cleanup"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350497    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06878ac5-98fb-4b1c-8486-a8486ec786d3" containerName="cleanup"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350502    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="06878ac5-98fb-4b1c-8486-a8486ec786d3" containerName="cleanup"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350510    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261c76ab-f598-4b3f-8ab7-8e11a4445510" containerName="payload-pre-processing"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350515    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="261c76ab-f598-4b3f-8ab7-8e11a4445510" containerName="payload-pre-processing"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350525    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" containerName="maas-api"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350530    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" containerName="maas-api"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350587    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0cf9024-16f3-407b-a19f-650a093389b5" containerName="payload-processing"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350595    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="25282b06-1c47-4813-ade6-5c75003d69fa" containerName="cleanup"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350603    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="261c76ab-f598-4b3f-8ab7-8e11a4445510" containerName="payload-pre-processing"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350608    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" containerName="maas-api"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350613    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="06878ac5-98fb-4b1c-8486-a8486ec786d3" containerName="cleanup"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350619    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="19753b29-4cfc-45da-b91a-73fd693b1d25" containerName="payload-processing"
Sep 14 08:15:55.350624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.350623    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5888666f-b50a-47e7-bec6-14a497b29138" containerName="payload-pre-processing"
Sep 14 08:15:55.353392 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.353375    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:55.355505 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.355486    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:15:55.360754 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.360729    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"]
Sep 14 08:15:55.433809 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.433775    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7238df32-c26f-4bb3-9ae7-75768ec3e221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj\" (UID: \"7238df32-c26f-4bb3-9ae7-75768ec3e221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:55.534451 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.534414    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7238df32-c26f-4bb3-9ae7-75768ec3e221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj\" (UID: \"7238df32-c26f-4bb3-9ae7-75768ec3e221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:55.535110 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.535086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7238df32-c26f-4bb3-9ae7-75768ec3e221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj\" (UID: \"7238df32-c26f-4bb3-9ae7-75768ec3e221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:55.665105 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.665012    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:55.936352 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:55.936322    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"]
Sep 14 08:15:55.937417 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:15:55.937395    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7238df32_c26f_4bb3_9ae7_75768ec3e221.slice/crio-3ce818a304ae0540d72e8e6dbaa73e754bfa7c2f7c17acf417431c5e8a486faf WatchSource:0}: Error finding container 3ce818a304ae0540d72e8e6dbaa73e754bfa7c2f7c17acf417431c5e8a486faf: Status 404 returned error can't find the container with id 3ce818a304ae0540d72e8e6dbaa73e754bfa7c2f7c17acf417431c5e8a486faf
Sep 14 08:15:56.018680 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:56.018645    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19753b29-4cfc-45da-b91a-73fd693b1d25" path="/var/lib/kubelet/pods/19753b29-4cfc-45da-b91a-73fd693b1d25/volumes"
Sep 14 08:15:56.019048 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:56.019033    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261c76ab-f598-4b3f-8ab7-8e11a4445510" path="/var/lib/kubelet/pods/261c76ab-f598-4b3f-8ab7-8e11a4445510/volumes"
Sep 14 08:15:56.019415 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:56.019403    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" path="/var/lib/kubelet/pods/40c2e61e-66dd-4fb8-92d6-ef60c3f418e5/volumes"
Sep 14 08:15:56.465003 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:56.464949    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj" event={"ID":"7238df32-c26f-4bb3-9ae7-75768ec3e221","Type":"ContainerStarted","Data":"82465d862c670b04ff4d5a07a358e620b33b64e61118cfdb9896e43263f90bea"}
Sep 14 08:15:56.465350 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:56.465028    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj" event={"ID":"7238df32-c26f-4bb3-9ae7-75768ec3e221","Type":"ContainerStarted","Data":"3ce818a304ae0540d72e8e6dbaa73e754bfa7c2f7c17acf417431c5e8a486faf"}
Sep 14 08:15:56.479795 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:56.479746    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj" podStartSLOduration=1.4797287670000001 podStartE2EDuration="1.479728767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:56.477731569 +0000 UTC m=+1612.977124366" watchObservedRunningTime="2026-09-14 08:15:56.479728767 +0000 UTC m=+1612.979121549"
Sep 14 08:15:57.059491 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:57.059437    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e721c-84b78cfb65-ljp5c" podUID="40c2e61e-66dd-4fb8-92d6-ef60c3f418e5" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.73:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:15:57.472095 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:57.472060    2556 generic.go:356] "Generic (PLEG): container finished" podID="7238df32-c26f-4bb3-9ae7-75768ec3e221" containerID="82465d862c670b04ff4d5a07a358e620b33b64e61118cfdb9896e43263f90bea" exitCode=0
Sep 14 08:15:57.472477 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:57.472152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj" event={"ID":"7238df32-c26f-4bb3-9ae7-75768ec3e221","Type":"ContainerDied","Data":"82465d862c670b04ff4d5a07a358e620b33b64e61118cfdb9896e43263f90bea"}
Sep 14 08:15:58.605316 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:58.605281    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:58.765905 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:58.765808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7238df32-c26f-4bb3-9ae7-75768ec3e221-maas-api-service-ca\") pod \"7238df32-c26f-4bb3-9ae7-75768ec3e221\" (UID: \"7238df32-c26f-4bb3-9ae7-75768ec3e221\") "
Sep 14 08:15:58.766233 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:58.766210    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7238df32-c26f-4bb3-9ae7-75768ec3e221-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7238df32-c26f-4bb3-9ae7-75768ec3e221" (UID: "7238df32-c26f-4bb3-9ae7-75768ec3e221"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:58.867127 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:58.867087    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7238df32-c26f-4bb3-9ae7-75768ec3e221-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:15:59.485461 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:59.485428    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"
Sep 14 08:15:59.485610 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:59.485460    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj" event={"ID":"7238df32-c26f-4bb3-9ae7-75768ec3e221","Type":"ContainerDied","Data":"3ce818a304ae0540d72e8e6dbaa73e754bfa7c2f7c17acf417431c5e8a486faf"}
Sep 14 08:15:59.485610 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:15:59.485506    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ce818a304ae0540d72e8e6dbaa73e754bfa7c2f7c17acf417431c5e8a486faf"
Sep 14 08:16:00.310615 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:00.310584    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"]
Sep 14 08:16:00.330866 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:00.330807    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"]
Sep 14 08:16:01.178475 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.178443    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"]
Sep 14 08:16:01.178826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.178814    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7238df32-c26f-4bb3-9ae7-75768ec3e221" containerName="revoke-keys"
Sep 14 08:16:01.178876 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.178827    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7238df32-c26f-4bb3-9ae7-75768ec3e221" containerName="revoke-keys"
Sep 14 08:16:01.178912 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.178904    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7238df32-c26f-4bb3-9ae7-75768ec3e221" containerName="revoke-keys"
Sep 14 08:16:01.180984 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.180951    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:01.182967 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.182947    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:16:01.189750 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.189214    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"]
Sep 14 08:16:01.290332 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.290297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ee2992-66e7-4ab6-a785-effadac82b5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp\" (UID: \"c5ee2992-66e7-4ab6-a785-effadac82b5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:01.391728 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.391687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ee2992-66e7-4ab6-a785-effadac82b5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp\" (UID: \"c5ee2992-66e7-4ab6-a785-effadac82b5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:01.392358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.392331    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ee2992-66e7-4ab6-a785-effadac82b5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp\" (UID: \"c5ee2992-66e7-4ab6-a785-effadac82b5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:01.493719 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.493630    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:01.766286 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:01.766249    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"]
Sep 14 08:16:01.767605 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:16:01.767577    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ee2992_66e7_4ab6_a785_effadac82b5e.slice/crio-ae0742888062b34fffcd3d722259f07498381e2be5d3542728d24a4677e14fd0 WatchSource:0}: Error finding container ae0742888062b34fffcd3d722259f07498381e2be5d3542728d24a4677e14fd0: Status 404 returned error can't find the container with id ae0742888062b34fffcd3d722259f07498381e2be5d3542728d24a4677e14fd0
Sep 14 08:16:02.505703 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:02.505665    2556 generic.go:356] "Generic (PLEG): container finished" podID="c5ee2992-66e7-4ab6-a785-effadac82b5e" containerID="6e461bcdb6f31a8d3695fc78e33af3be922b4004aeb6c1dec9e0ed38095dd0f3" exitCode=0
Sep 14 08:16:02.506161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:02.505728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp" event={"ID":"c5ee2992-66e7-4ab6-a785-effadac82b5e","Type":"ContainerDied","Data":"6e461bcdb6f31a8d3695fc78e33af3be922b4004aeb6c1dec9e0ed38095dd0f3"}
Sep 14 08:16:02.506161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:02.505789    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp" event={"ID":"c5ee2992-66e7-4ab6-a785-effadac82b5e","Type":"ContainerStarted","Data":"ae0742888062b34fffcd3d722259f07498381e2be5d3542728d24a4677e14fd0"}
Sep 14 08:16:03.649799 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:03.649773    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:03.814768 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:03.814683    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ee2992-66e7-4ab6-a785-effadac82b5e-maas-api-service-ca\") pod \"c5ee2992-66e7-4ab6-a785-effadac82b5e\" (UID: \"c5ee2992-66e7-4ab6-a785-effadac82b5e\") "
Sep 14 08:16:03.815129 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:03.815098    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ee2992-66e7-4ab6-a785-effadac82b5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5ee2992-66e7-4ab6-a785-effadac82b5e" (UID: "c5ee2992-66e7-4ab6-a785-effadac82b5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:03.915991 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:03.915949    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ee2992-66e7-4ab6-a785-effadac82b5e-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:04.517769 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:04.517679    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"
Sep 14 08:16:04.517769 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:04.517722    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp" event={"ID":"c5ee2992-66e7-4ab6-a785-effadac82b5e","Type":"ContainerDied","Data":"ae0742888062b34fffcd3d722259f07498381e2be5d3542728d24a4677e14fd0"}
Sep 14 08:16:04.517769 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:04.517771    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae0742888062b34fffcd3d722259f07498381e2be5d3542728d24a4677e14fd0"
Sep 14 08:16:11.615037 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.615002    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"]
Sep 14 08:16:11.615768 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.615730    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4" podUID="c417be04-938b-4acf-a7eb-91fe8e7fa091" containerName="maas-api" containerID="cri-o://2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf" gracePeriod=30
Sep 14 08:16:11.676588 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.676550    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"]
Sep 14 08:16:11.676869 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.676819    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n" podUID="615d2e12-74e3-47d9-876d-94a34276f5c3" containerName="payload-processing" containerID="cri-o://c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7" gracePeriod=30
Sep 14 08:16:11.863290 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.863223    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"]
Sep 14 08:16:11.867375 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.867276    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd7582-4lwhj"]
Sep 14 08:16:11.944825 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.944794    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:16:11.982233 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.982210    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:16:11.984424 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984404    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417be04-938b-4acf-a7eb-91fe8e7fa091-openshift-service-ca\") pod \"c417be04-938b-4acf-a7eb-91fe8e7fa091\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") "
Sep 14 08:16:11.984534 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984475    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txxrj\" (UniqueName: \"kubernetes.io/projected/615d2e12-74e3-47d9-876d-94a34276f5c3-kube-api-access-txxrj\") pod \"615d2e12-74e3-47d9-876d-94a34276f5c3\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") "
Sep 14 08:16:11.984534 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984515    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-metrics-certs\") pod \"c417be04-938b-4acf-a7eb-91fe8e7fa091\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") "
Sep 14 08:16:11.984646 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984555    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk2qv\" (UniqueName: \"kubernetes.io/projected/c417be04-938b-4acf-a7eb-91fe8e7fa091-kube-api-access-hk2qv\") pod \"c417be04-938b-4acf-a7eb-91fe8e7fa091\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") "
Sep 14 08:16:11.984646 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984602    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-maas-api-tls\") pod \"c417be04-938b-4acf-a7eb-91fe8e7fa091\" (UID: \"c417be04-938b-4acf-a7eb-91fe8e7fa091\") "
Sep 14 08:16:11.984646 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984635    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615d2e12-74e3-47d9-876d-94a34276f5c3-plugins-config-volume\") pod \"615d2e12-74e3-47d9-876d-94a34276f5c3\" (UID: \"615d2e12-74e3-47d9-876d-94a34276f5c3\") "
Sep 14 08:16:11.984811 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984754    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c417be04-938b-4acf-a7eb-91fe8e7fa091-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c417be04-938b-4acf-a7eb-91fe8e7fa091" (UID: "c417be04-938b-4acf-a7eb-91fe8e7fa091"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:11.984985 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.984945    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417be04-938b-4acf-a7eb-91fe8e7fa091-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:11.985313 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.985239    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615d2e12-74e3-47d9-876d-94a34276f5c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "615d2e12-74e3-47d9-876d-94a34276f5c3" (UID: "615d2e12-74e3-47d9-876d-94a34276f5c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:11.986561 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.986539    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c417be04-938b-4acf-a7eb-91fe8e7fa091" (UID: "c417be04-938b-4acf-a7eb-91fe8e7fa091"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:11.987061 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.987030    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615d2e12-74e3-47d9-876d-94a34276f5c3-kube-api-access-txxrj" (OuterVolumeSpecName: "kube-api-access-txxrj") pod "615d2e12-74e3-47d9-876d-94a34276f5c3" (UID: "615d2e12-74e3-47d9-876d-94a34276f5c3"). InnerVolumeSpecName "kube-api-access-txxrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:11.987148 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.987087    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c417be04-938b-4acf-a7eb-91fe8e7fa091-kube-api-access-hk2qv" (OuterVolumeSpecName: "kube-api-access-hk2qv") pod "c417be04-938b-4acf-a7eb-91fe8e7fa091" (UID: "c417be04-938b-4acf-a7eb-91fe8e7fa091"). InnerVolumeSpecName "kube-api-access-hk2qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:11.987268 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:11.987250    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c417be04-938b-4acf-a7eb-91fe8e7fa091" (UID: "c417be04-938b-4acf-a7eb-91fe8e7fa091"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:12.019453 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.019419    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7238df32-c26f-4bb3-9ae7-75768ec3e221" path="/var/lib/kubelet/pods/7238df32-c26f-4bb3-9ae7-75768ec3e221/volumes"
Sep 14 08:16:12.085453 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.085415    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txxrj\" (UniqueName: \"kubernetes.io/projected/615d2e12-74e3-47d9-876d-94a34276f5c3-kube-api-access-txxrj\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.085453 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.085444    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.085453 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.085453    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk2qv\" (UniqueName: \"kubernetes.io/projected/c417be04-938b-4acf-a7eb-91fe8e7fa091-kube-api-access-hk2qv\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.085662 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.085462    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c417be04-938b-4acf-a7eb-91fe8e7fa091-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.085662 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.085471    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615d2e12-74e3-47d9-876d-94a34276f5c3-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.175664 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.175629    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"]
Sep 14 08:16:12.178016 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.177993    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b8346-7zfwp"]
Sep 14 08:16:12.567225 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.567126    2556 generic.go:356] "Generic (PLEG): container finished" podID="615d2e12-74e3-47d9-876d-94a34276f5c3" containerID="c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7" exitCode=0
Sep 14 08:16:12.567225 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.567182    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"
Sep 14 08:16:12.567225 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.567210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n" event={"ID":"615d2e12-74e3-47d9-876d-94a34276f5c3","Type":"ContainerDied","Data":"c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7"}
Sep 14 08:16:12.567499 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.567261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n" event={"ID":"615d2e12-74e3-47d9-876d-94a34276f5c3","Type":"ContainerDied","Data":"4df17c33047ac4ae52aa19cbaf64046f19bf3afcb296520a677b614ecf71f188"}
Sep 14 08:16:12.567499 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.567290    2556 scope.go:117] "RemoveContainer" containerID="c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7"
Sep 14 08:16:12.568694 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.568669    2556 generic.go:356] "Generic (PLEG): container finished" podID="c417be04-938b-4acf-a7eb-91fe8e7fa091" containerID="2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf" exitCode=0
Sep 14 08:16:12.568810 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.568717    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"
Sep 14 08:16:12.568810 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.568761    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4" event={"ID":"c417be04-938b-4acf-a7eb-91fe8e7fa091","Type":"ContainerDied","Data":"2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf"}
Sep 14 08:16:12.568902 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.568816    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4" event={"ID":"c417be04-938b-4acf-a7eb-91fe8e7fa091","Type":"ContainerDied","Data":"52105d29ebb1d933530b48d0be2ec4b884cee86fce58ef72fcbe7d44604ac073"}
Sep 14 08:16:12.576479 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.576469    2556 scope.go:117] "RemoveContainer" containerID="c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7"
Sep 14 08:16:12.576776 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:16:12.576752    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7\": container with ID starting with c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7 not found: ID does not exist" containerID="c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7"
Sep 14 08:16:12.576879 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.576787    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7"} err="failed to get container status \"c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7\": rpc error: code = NotFound desc = could not find container \"c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7\": container with ID starting with c7665cb63bb6165eec0b1760c818c1b452f792f0db7303c151160e3eb28f5cc7 not found: ID does not exist"
Sep 14 08:16:12.576879 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.576813    2556 scope.go:117] "RemoveContainer" containerID="2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf"
Sep 14 08:16:12.586082 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.586056    2556 scope.go:117] "RemoveContainer" containerID="2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf"
Sep 14 08:16:12.586354 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:16:12.586335    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf\": container with ID starting with 2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf not found: ID does not exist" containerID="2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf"
Sep 14 08:16:12.586415 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.586360    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf"} err="failed to get container status \"2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf\": rpc error: code = NotFound desc = could not find container \"2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf\": container with ID starting with 2c0d0574d7377a98afabf100cf679fa9067f44b5562f40a62ed5ed66556aafaf not found: ID does not exist"
Sep 14 08:16:12.586684 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.586673    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"]
Sep 14 08:16:12.589263 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.589245    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-cd7582-65b9f6cfbf-c445n"]
Sep 14 08:16:12.596567 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.596544    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"]
Sep 14 08:16:12.604007 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:12.603987    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd7582-ff7585cc-gd2j4"]
Sep 14 08:16:14.020481 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:14.020444    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615d2e12-74e3-47d9-876d-94a34276f5c3" path="/var/lib/kubelet/pods/615d2e12-74e3-47d9-876d-94a34276f5c3/volumes"
Sep 14 08:16:14.020899 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:14.020882    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c417be04-938b-4acf-a7eb-91fe8e7fa091" path="/var/lib/kubelet/pods/c417be04-938b-4acf-a7eb-91fe8e7fa091/volumes"
Sep 14 08:16:14.021206 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:14.021193    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ee2992-66e7-4ab6-a785-effadac82b5e" path="/var/lib/kubelet/pods/c5ee2992-66e7-4ab6-a785-effadac82b5e/volumes"
Sep 14 08:16:15.329705 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.329663    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n" podUID="2c39f433-5f93-487d-8035-74738268eb47" containerName="main" containerID="cri-o://b8d5a8bbd1f4287cdd75d6f12bb44f12e7ac52c967a062aff74983d53fef1c13" gracePeriod=45
Sep 14 08:16:15.349385 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.349347    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw" podUID="3d34f517-a2b6-4a16-a806-b56716b940f0" containerName="main" containerID="cri-o://479b2161f2325c0bd4d25130c33e373a780aaa7a6b7c1a95909dbd6b762149fd" gracePeriod=45
Sep 14 08:16:15.598872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.598802    2556 generic.go:356] "Generic (PLEG): container finished" podID="2c39f433-5f93-487d-8035-74738268eb47" containerID="b8d5a8bbd1f4287cdd75d6f12bb44f12e7ac52c967a062aff74983d53fef1c13" exitCode=0
Sep 14 08:16:15.599065 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.598876    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n" event={"ID":"2c39f433-5f93-487d-8035-74738268eb47","Type":"ContainerDied","Data":"b8d5a8bbd1f4287cdd75d6f12bb44f12e7ac52c967a062aff74983d53fef1c13"}
Sep 14 08:16:15.600763 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.600736    2556 generic.go:356] "Generic (PLEG): container finished" podID="3d34f517-a2b6-4a16-a806-b56716b940f0" containerID="479b2161f2325c0bd4d25130c33e373a780aaa7a6b7c1a95909dbd6b762149fd" exitCode=0
Sep 14 08:16:15.600896 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.600818    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw" event={"ID":"3d34f517-a2b6-4a16-a806-b56716b940f0","Type":"ContainerDied","Data":"479b2161f2325c0bd4d25130c33e373a780aaa7a6b7c1a95909dbd6b762149fd"}
Sep 14 08:16:15.625653 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.625630    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:16:15.628905 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.628874    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:16:15.713656 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713624    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-home\") pod \"2c39f433-5f93-487d-8035-74738268eb47\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") "
Sep 14 08:16:15.713824 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713683    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d34f517-a2b6-4a16-a806-b56716b940f0-tls-certs\") pod \"3d34f517-a2b6-4a16-a806-b56716b940f0\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") "
Sep 14 08:16:15.713824 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713714    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-dshm\") pod \"3d34f517-a2b6-4a16-a806-b56716b940f0\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") "
Sep 14 08:16:15.713824 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713743    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-model-cache\") pod \"2c39f433-5f93-487d-8035-74738268eb47\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") "
Sep 14 08:16:15.713824 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713767    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz578\" (UniqueName: \"kubernetes.io/projected/2c39f433-5f93-487d-8035-74738268eb47-kube-api-access-gz578\") pod \"2c39f433-5f93-487d-8035-74738268eb47\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713833    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-model-cache\") pod \"3d34f517-a2b6-4a16-a806-b56716b940f0\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713868    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c39f433-5f93-487d-8035-74738268eb47-tls-certs\") pod \"2c39f433-5f93-487d-8035-74738268eb47\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713898    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-home\") pod \"3d34f517-a2b6-4a16-a806-b56716b940f0\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713922    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-dshm\") pod \"2c39f433-5f93-487d-8035-74738268eb47\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713941    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-home" (OuterVolumeSpecName: "home") pod "2c39f433-5f93-487d-8035-74738268eb47" (UID: "2c39f433-5f93-487d-8035-74738268eb47"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.713986    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfqdr\" (UniqueName: \"kubernetes.io/projected/3d34f517-a2b6-4a16-a806-b56716b940f0-kube-api-access-nfqdr\") pod \"3d34f517-a2b6-4a16-a806-b56716b940f0\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714017    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-tmp-dir\") pod \"3d34f517-a2b6-4a16-a806-b56716b940f0\" (UID: \"3d34f517-a2b6-4a16-a806-b56716b940f0\") "
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714018    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c39f433-5f93-487d-8035-74738268eb47" (UID: "2c39f433-5f93-487d-8035-74738268eb47"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.714080 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714045    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-tmp-dir\") pod \"2c39f433-5f93-487d-8035-74738268eb47\" (UID: \"2c39f433-5f93-487d-8035-74738268eb47\") "
Sep 14 08:16:15.714523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714182    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-home" (OuterVolumeSpecName: "home") pod "3d34f517-a2b6-4a16-a806-b56716b940f0" (UID: "3d34f517-a2b6-4a16-a806-b56716b940f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.714523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714393    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d34f517-a2b6-4a16-a806-b56716b940f0" (UID: "3d34f517-a2b6-4a16-a806-b56716b940f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.714523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714405    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c39f433-5f93-487d-8035-74738268eb47" (UID: "2c39f433-5f93-487d-8035-74738268eb47"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.714523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714400    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.714523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714446    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.714523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714460    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.714784 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.714611    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d34f517-a2b6-4a16-a806-b56716b940f0" (UID: "3d34f517-a2b6-4a16-a806-b56716b940f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.715983 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.715946    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-dshm" (OuterVolumeSpecName: "dshm") pod "3d34f517-a2b6-4a16-a806-b56716b940f0" (UID: "3d34f517-a2b6-4a16-a806-b56716b940f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.716367 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.716334    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c39f433-5f93-487d-8035-74738268eb47-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c39f433-5f93-487d-8035-74738268eb47" (UID: "2c39f433-5f93-487d-8035-74738268eb47"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:15.716367 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.716359    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-dshm" (OuterVolumeSpecName: "dshm") pod "2c39f433-5f93-487d-8035-74738268eb47" (UID: "2c39f433-5f93-487d-8035-74738268eb47"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.716522 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.716384    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d34f517-a2b6-4a16-a806-b56716b940f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d34f517-a2b6-4a16-a806-b56716b940f0" (UID: "3d34f517-a2b6-4a16-a806-b56716b940f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:15.716522 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.716426    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c39f433-5f93-487d-8035-74738268eb47-kube-api-access-gz578" (OuterVolumeSpecName: "kube-api-access-gz578") pod "2c39f433-5f93-487d-8035-74738268eb47" (UID: "2c39f433-5f93-487d-8035-74738268eb47"). InnerVolumeSpecName "kube-api-access-gz578". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:15.716522 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.716511    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d34f517-a2b6-4a16-a806-b56716b940f0-kube-api-access-nfqdr" (OuterVolumeSpecName: "kube-api-access-nfqdr") pod "3d34f517-a2b6-4a16-a806-b56716b940f0" (UID: "3d34f517-a2b6-4a16-a806-b56716b940f0"). InnerVolumeSpecName "kube-api-access-nfqdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:15.815005 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.814942    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d34f517-a2b6-4a16-a806-b56716b940f0-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815005 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.814982    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815005 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.814996    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz578\" (UniqueName: \"kubernetes.io/projected/2c39f433-5f93-487d-8035-74738268eb47-kube-api-access-gz578\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815005 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.815011    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815274 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.815022    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c39f433-5f93-487d-8035-74738268eb47-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815274 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.815029    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815274 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.815037    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfqdr\" (UniqueName: \"kubernetes.io/projected/3d34f517-a2b6-4a16-a806-b56716b940f0-kube-api-access-nfqdr\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815274 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.815065    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d34f517-a2b6-4a16-a806-b56716b940f0-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.815274 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:15.815083    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c39f433-5f93-487d-8035-74738268eb47-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.607643 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.607608    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"
Sep 14 08:16:16.608098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.607618    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n" event={"ID":"2c39f433-5f93-487d-8035-74738268eb47","Type":"ContainerDied","Data":"b605e181761f3fa76acdf48f0cdc598181690cae42a61b111d3e914ad7f514f7"}
Sep 14 08:16:16.608098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.607760    2556 scope.go:117] "RemoveContainer" containerID="b8d5a8bbd1f4287cdd75d6f12bb44f12e7ac52c967a062aff74983d53fef1c13"
Sep 14 08:16:16.609296 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.609274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw" event={"ID":"3d34f517-a2b6-4a16-a806-b56716b940f0","Type":"ContainerDied","Data":"254b76b91b4bca93cfc3e6f3f1498ccd4e15aeb47917384b571ac46759e2d1dc"}
Sep 14 08:16:16.609422 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.609278    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"
Sep 14 08:16:16.617090 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.617068    2556 scope.go:117] "RemoveContainer" containerID="479b2161f2325c0bd4d25130c33e373a780aaa7a6b7c1a95909dbd6b762149fd"
Sep 14 08:16:16.626065 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.626045    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"]
Sep 14 08:16:16.628509 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.628490    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-w4-cd7582-kserve-874ccd59d-cn54n"]
Sep 14 08:16:16.636905 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.636886    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"]
Sep 14 08:16:16.640416 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.640393    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/simulator-w4-cd7582-kserve-6d9b4c8b66-wzhmw"]
Sep 14 08:16:16.648304 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648282    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-522adb"]
Sep 14 08:16:16.648643 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648628    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ee2992-66e7-4ab6-a785-effadac82b5e" containerName="revoke-keys"
Sep 14 08:16:16.648711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648647    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ee2992-66e7-4ab6-a785-effadac82b5e" containerName="revoke-keys"
Sep 14 08:16:16.648711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648665    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c39f433-5f93-487d-8035-74738268eb47" containerName="main"
Sep 14 08:16:16.648711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648673    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c39f433-5f93-487d-8035-74738268eb47" containerName="main"
Sep 14 08:16:16.648711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648692    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c417be04-938b-4acf-a7eb-91fe8e7fa091" containerName="maas-api"
Sep 14 08:16:16.648711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648700    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c417be04-938b-4acf-a7eb-91fe8e7fa091" containerName="maas-api"
Sep 14 08:16:16.648711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648712    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d34f517-a2b6-4a16-a806-b56716b940f0" containerName="main"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648720    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d34f517-a2b6-4a16-a806-b56716b940f0" containerName="main"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648740    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615d2e12-74e3-47d9-876d-94a34276f5c3" containerName="payload-processing"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648748    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="615d2e12-74e3-47d9-876d-94a34276f5c3" containerName="payload-processing"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648836    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c39f433-5f93-487d-8035-74738268eb47" containerName="main"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648847    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d34f517-a2b6-4a16-a806-b56716b940f0" containerName="main"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648858    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c417be04-938b-4acf-a7eb-91fe8e7fa091" containerName="maas-api"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648867    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="615d2e12-74e3-47d9-876d-94a34276f5c3" containerName="payload-processing"
Sep 14 08:16:16.649014 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.648879    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ee2992-66e7-4ab6-a785-effadac82b5e" containerName="revoke-keys"
Sep 14 08:16:16.653466 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.653447    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:16.655627 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.655530    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qcf9c\""
Sep 14 08:16:16.657478 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.657457    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-522adb"]
Sep 14 08:16:16.722740 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.722705    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nndx\" (UniqueName: \"kubernetes.io/projected/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770-kube-api-access-4nndx\") pod \"test-curl-7752-522adb\" (UID: \"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770\") " pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:16.823488 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.823454    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nndx\" (UniqueName: \"kubernetes.io/projected/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770-kube-api-access-4nndx\") pod \"test-curl-7752-522adb\" (UID: \"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770\") " pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:16.830472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.830443    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nndx\" (UniqueName: \"kubernetes.io/projected/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770-kube-api-access-4nndx\") pod \"test-curl-7752-522adb\" (UID: \"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770\") " pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:16.963774 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:16.963743    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:17.232814 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:17.232787    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-522adb"]
Sep 14 08:16:17.234438 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:16:17.234411    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93b5a247_2a0f_4a6f_94ba_78fb3f8f0770.slice/crio-ee2012668ce2f28b3bdcbbf6e0fd532bdab266365285edae1c54949c06d79681 WatchSource:0}: Error finding container ee2012668ce2f28b3bdcbbf6e0fd532bdab266365285edae1c54949c06d79681: Status 404 returned error can't find the container with id ee2012668ce2f28b3bdcbbf6e0fd532bdab266365285edae1c54949c06d79681
Sep 14 08:16:17.619506 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:17.619416    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-522adb" event={"ID":"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770","Type":"ContainerStarted","Data":"4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49"}
Sep 14 08:16:17.619506 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:17.619467    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-522adb" event={"ID":"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770","Type":"ContainerStarted","Data":"ee2012668ce2f28b3bdcbbf6e0fd532bdab266365285edae1c54949c06d79681"}
Sep 14 08:16:17.634818 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:17.634757    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-522adb" podStartSLOduration=1.63473603 podStartE2EDuration="1.63473603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:17.633574361 +0000 UTC m=+1634.132967138" watchObservedRunningTime="2026-09-14 08:16:17.63473603 +0000 UTC m=+1634.134128820"
Sep 14 08:16:18.019599 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:18.019564    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c39f433-5f93-487d-8035-74738268eb47" path="/var/lib/kubelet/pods/2c39f433-5f93-487d-8035-74738268eb47/volumes"
Sep 14 08:16:18.019957 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:18.019944    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d34f517-a2b6-4a16-a806-b56716b940f0" path="/var/lib/kubelet/pods/3d34f517-a2b6-4a16-a806-b56716b940f0/volumes"
Sep 14 08:16:18.249958 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:18.249922    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-522adb"]
Sep 14 08:16:18.253864 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:18.253835    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-522adb"]
Sep 14 08:16:18.625480 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:16:18.625441    2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-522adb" container="test-curl-7752-522adb"
Sep 14 08:16:19.630536 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:19.630484    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-522adb" podUID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" containerName="test-curl-7752-522adb" containerID="cri-o://4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49" gracePeriod=2
Sep 14 08:16:19.632465 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:16:19.632430    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-522adb\" is forbidden: User \"system:node:ip-10-0-131-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-236.ec2.internal' and this object" podUID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:20.181347 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.181299    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"]
Sep 14 08:16:20.181701 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.181686    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" containerName="test-curl-7752-522adb"
Sep 14 08:16:20.181701 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.181701    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" containerName="test-curl-7752-522adb"
Sep 14 08:16:20.181838 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.181776    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" containerName="test-curl-7752-522adb"
Sep 14 08:16:20.185932 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.185911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:20.188146 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.188123    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:16:20.192262 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.192241    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"]
Sep 14 08:16:20.254192 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.254157    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/188dc2ce-2bd0-491d-9405-fb9c420ee9f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n\" (UID: \"188dc2ce-2bd0-491d-9405-fb9c420ee9f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:20.354796 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.354758    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/188dc2ce-2bd0-491d-9405-fb9c420ee9f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n\" (UID: \"188dc2ce-2bd0-491d-9405-fb9c420ee9f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:20.355460 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.355437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/188dc2ce-2bd0-491d-9405-fb9c420ee9f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n\" (UID: \"188dc2ce-2bd0-491d-9405-fb9c420ee9f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:20.498233 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.498158    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:20.772590 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:20.772564    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"]
Sep 14 08:16:20.774069 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:16:20.774044    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188dc2ce_2bd0_491d_9405_fb9c420ee9f4.slice/crio-ec08b1a74760536cb01a39a329d070c8147e0b64eb26966de5d7ddd323dff9e1 WatchSource:0}: Error finding container ec08b1a74760536cb01a39a329d070c8147e0b64eb26966de5d7ddd323dff9e1: Status 404 returned error can't find the container with id ec08b1a74760536cb01a39a329d070c8147e0b64eb26966de5d7ddd323dff9e1
Sep 14 08:16:21.644113 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.644083    2556 generic.go:356] "Generic (PLEG): container finished" podID="188dc2ce-2bd0-491d-9405-fb9c420ee9f4" containerID="b2f5e2a66ec5e32c91374d3d58b57d4e4115530ab9f20ddd42e04fc87e022828" exitCode=0
Sep 14 08:16:21.644312 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.644168    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n" event={"ID":"188dc2ce-2bd0-491d-9405-fb9c420ee9f4","Type":"ContainerDied","Data":"b2f5e2a66ec5e32c91374d3d58b57d4e4115530ab9f20ddd42e04fc87e022828"}
Sep 14 08:16:21.644312 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.644226    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n" event={"ID":"188dc2ce-2bd0-491d-9405-fb9c420ee9f4","Type":"ContainerStarted","Data":"ec08b1a74760536cb01a39a329d070c8147e0b64eb26966de5d7ddd323dff9e1"}
Sep 14 08:16:21.819681 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.819653    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:21.868247 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.868218    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nndx\" (UniqueName: \"kubernetes.io/projected/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770-kube-api-access-4nndx\") pod \"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770\" (UID: \"93b5a247-2a0f-4a6f-94ba-78fb3f8f0770\") "
Sep 14 08:16:21.870398 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.870365    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770-kube-api-access-4nndx" (OuterVolumeSpecName: "kube-api-access-4nndx") pod "93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" (UID: "93b5a247-2a0f-4a6f-94ba-78fb3f8f0770"). InnerVolumeSpecName "kube-api-access-4nndx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:21.969780 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:21.969743    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nndx\" (UniqueName: \"kubernetes.io/projected/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770-kube-api-access-4nndx\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:22.018736 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.018659    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" path="/var/lib/kubelet/pods/93b5a247-2a0f-4a6f-94ba-78fb3f8f0770/volumes"
Sep 14 08:16:22.650793 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.650762    2556 generic.go:356] "Generic (PLEG): container finished" podID="93b5a247-2a0f-4a6f-94ba-78fb3f8f0770" containerID="4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49" exitCode=137
Sep 14 08:16:22.650958 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.650809    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-522adb"
Sep 14 08:16:22.650958 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.650839    2556 scope.go:117] "RemoveContainer" containerID="4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49"
Sep 14 08:16:22.660728 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.660693    2556 scope.go:117] "RemoveContainer" containerID="4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49"
Sep 14 08:16:22.661037 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:16:22.661017    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49\": container with ID starting with 4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49 not found: ID does not exist" containerID="4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49"
Sep 14 08:16:22.661103 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.661053    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49"} err="failed to get container status \"4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49\": rpc error: code = NotFound desc = could not find container \"4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49\": container with ID starting with 4b2bac3a6ff75b1de173faf0e8ca63a43c56f96c34fa3f8a913c75467e5e9a49 not found: ID does not exist"
Sep 14 08:16:22.788929 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.788906    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:22.879011 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.878956    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/188dc2ce-2bd0-491d-9405-fb9c420ee9f4-maas-api-service-ca\") pod \"188dc2ce-2bd0-491d-9405-fb9c420ee9f4\" (UID: \"188dc2ce-2bd0-491d-9405-fb9c420ee9f4\") "
Sep 14 08:16:22.879388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.879265    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/188dc2ce-2bd0-491d-9405-fb9c420ee9f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "188dc2ce-2bd0-491d-9405-fb9c420ee9f4" (UID: "188dc2ce-2bd0-491d-9405-fb9c420ee9f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:22.980271 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:22.980234    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/188dc2ce-2bd0-491d-9405-fb9c420ee9f4-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:23.658705 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:23.658677    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"
Sep 14 08:16:23.658886 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:23.658706    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n" event={"ID":"188dc2ce-2bd0-491d-9405-fb9c420ee9f4","Type":"ContainerDied","Data":"ec08b1a74760536cb01a39a329d070c8147e0b64eb26966de5d7ddd323dff9e1"}
Sep 14 08:16:23.658886 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:23.658755    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec08b1a74760536cb01a39a329d070c8147e0b64eb26966de5d7ddd323dff9e1"
Sep 14 08:16:25.779881 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:25.779845    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"]
Sep 14 08:16:30.814215 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:30.814132    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"]
Sep 14 08:16:30.815666 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:30.815634    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4d8f28-nrh8n"]
Sep 14 08:16:32.018381 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:32.018340    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="188dc2ce-2bd0-491d-9405-fb9c420ee9f4" path="/var/lib/kubelet/pods/188dc2ce-2bd0-491d-9405-fb9c420ee9f4/volumes"
Sep 14 08:16:40.799405 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:40.799365    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk" podUID="e638c228-8984-4172-ae67-34daa9fa0db1" containerName="main" containerID="cri-o://46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa" gracePeriod=45
Sep 14 08:16:41.057319 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.057253    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:16:41.144851 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.144812    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-model-cache\") pod \"e638c228-8984-4172-ae67-34daa9fa0db1\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") "
Sep 14 08:16:41.144851 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.144854    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d59r6\" (UniqueName: \"kubernetes.io/projected/e638c228-8984-4172-ae67-34daa9fa0db1-kube-api-access-d59r6\") pod \"e638c228-8984-4172-ae67-34daa9fa0db1\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") "
Sep 14 08:16:41.145078 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.144894    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-home\") pod \"e638c228-8984-4172-ae67-34daa9fa0db1\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") "
Sep 14 08:16:41.145078 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.144925    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-tmp-dir\") pod \"e638c228-8984-4172-ae67-34daa9fa0db1\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") "
Sep 14 08:16:41.145078 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145054    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e638c228-8984-4172-ae67-34daa9fa0db1-tls-certs\") pod \"e638c228-8984-4172-ae67-34daa9fa0db1\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") "
Sep 14 08:16:41.145200 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145085    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-dshm\") pod \"e638c228-8984-4172-ae67-34daa9fa0db1\" (UID: \"e638c228-8984-4172-ae67-34daa9fa0db1\") "
Sep 14 08:16:41.145200 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145156    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-model-cache" (OuterVolumeSpecName: "model-cache") pod "e638c228-8984-4172-ae67-34daa9fa0db1" (UID: "e638c228-8984-4172-ae67-34daa9fa0db1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:41.145286 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145173    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-home" (OuterVolumeSpecName: "home") pod "e638c228-8984-4172-ae67-34daa9fa0db1" (UID: "e638c228-8984-4172-ae67-34daa9fa0db1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:41.145286 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145240    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e638c228-8984-4172-ae67-34daa9fa0db1" (UID: "e638c228-8984-4172-ae67-34daa9fa0db1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:41.145462 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145446    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:41.145529 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145464    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:41.145529 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.145473    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:41.147212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.147183    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-dshm" (OuterVolumeSpecName: "dshm") pod "e638c228-8984-4172-ae67-34daa9fa0db1" (UID: "e638c228-8984-4172-ae67-34daa9fa0db1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:41.147212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.147196    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e638c228-8984-4172-ae67-34daa9fa0db1-kube-api-access-d59r6" (OuterVolumeSpecName: "kube-api-access-d59r6") pod "e638c228-8984-4172-ae67-34daa9fa0db1" (UID: "e638c228-8984-4172-ae67-34daa9fa0db1"). InnerVolumeSpecName "kube-api-access-d59r6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:41.147212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.147201    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e638c228-8984-4172-ae67-34daa9fa0db1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e638c228-8984-4172-ae67-34daa9fa0db1" (UID: "e638c228-8984-4172-ae67-34daa9fa0db1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:41.246221 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.246192    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e638c228-8984-4172-ae67-34daa9fa0db1-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:41.246221 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.246215    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e638c228-8984-4172-ae67-34daa9fa0db1-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:41.246394 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.246226    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d59r6\" (UniqueName: \"kubernetes.io/projected/e638c228-8984-4172-ae67-34daa9fa0db1-kube-api-access-d59r6\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:16:41.765633 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.765597    2556 generic.go:356] "Generic (PLEG): container finished" podID="e638c228-8984-4172-ae67-34daa9fa0db1" containerID="46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa" exitCode=0
Sep 14 08:16:41.765633 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.765633    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"
Sep 14 08:16:41.765906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.765679    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk" event={"ID":"e638c228-8984-4172-ae67-34daa9fa0db1","Type":"ContainerDied","Data":"46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa"}
Sep 14 08:16:41.765906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.765730    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk" event={"ID":"e638c228-8984-4172-ae67-34daa9fa0db1","Type":"ContainerDied","Data":"393ca29e7cca758cf1b3406ea717b85c0874eb00f4a251c6885570072673d4b7"}
Sep 14 08:16:41.765906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.765752    2556 scope.go:117] "RemoveContainer" containerID="46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa"
Sep 14 08:16:41.775345 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.775326    2556 scope.go:117] "RemoveContainer" containerID="46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa"
Sep 14 08:16:41.775602 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:16:41.775583    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa\": container with ID starting with 46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa not found: ID does not exist" containerID="46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa"
Sep 14 08:16:41.775663 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.775615    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa"} err="failed to get container status \"46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa\": rpc error: code = NotFound desc = could not find container \"46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa\": container with ID starting with 46ec15e161732e68808b8491b3117ce4e3f6eea78f580b5ce3ca0704f0ec2caa not found: ID does not exist"
Sep 14 08:16:41.786665 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.786638    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"]
Sep 14 08:16:41.790674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.790652    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/auth-test-model-fc0d6e-kserve-6cd95878c5-m5fnk"]
Sep 14 08:16:41.983562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.983526    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"]
Sep 14 08:16:41.983911 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.983898    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e638c228-8984-4172-ae67-34daa9fa0db1" containerName="main"
Sep 14 08:16:41.983911 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.983910    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e638c228-8984-4172-ae67-34daa9fa0db1" containerName="main"
Sep 14 08:16:41.984010 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.983927    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188dc2ce-2bd0-491d-9405-fb9c420ee9f4" containerName="revoke-keys"
Sep 14 08:16:41.984010 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.983933    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="188dc2ce-2bd0-491d-9405-fb9c420ee9f4" containerName="revoke-keys"
Sep 14 08:16:41.984091 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.984017    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="188dc2ce-2bd0-491d-9405-fb9c420ee9f4" containerName="revoke-keys"
Sep 14 08:16:41.984091 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.984031    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e638c228-8984-4172-ae67-34daa9fa0db1" containerName="main"
Sep 14 08:16:41.988289 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.988266    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:41.990608 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.990583    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"openshift-service-ca.crt\""
Sep 14 08:16:41.990608 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.990605    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"kube-root-ca.crt\""
Sep 14 08:16:41.990867 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.990848    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"sub-test-model-919cc9-kserve-self-signed-certs\""
Sep 14 08:16:41.990918 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.990903    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"default-dockercfg-7hm7s\""
Sep 14 08:16:41.996944 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:41.996919    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"]
Sep 14 08:16:42.019239 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.019179    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e638c228-8984-4172-ae67-34daa9fa0db1" path="/var/lib/kubelet/pods/e638c228-8984-4172-ae67-34daa9fa0db1/volumes"
Sep 14 08:16:42.052569 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.052545    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-home\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.052672 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.052579    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gccrk\" (UniqueName: \"kubernetes.io/projected/261b5e67-3607-4ea9-abf1-4626c72d3ce7-kube-api-access-gccrk\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.052672 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.052623    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tmp-dir\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.052672 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.052644    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-dshm\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.052780 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.052675    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.052780 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.052738    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-model-cache\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.153903 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.153870    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tmp-dir\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.153903 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.153904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-dshm\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154123 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.153988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154164 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.154136    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-model-cache\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154410 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.154345    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tmp-dir\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.154437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-model-cache\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.154523    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-home\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154684 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.154567    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gccrk\" (UniqueName: \"kubernetes.io/projected/261b5e67-3607-4ea9-abf1-4626c72d3ce7-kube-api-access-gccrk\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.154796 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.154775    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-home\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.156249 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.156217    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-dshm\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.156624 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.156605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.161854 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.161834    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gccrk\" (UniqueName: \"kubernetes.io/projected/261b5e67-3607-4ea9-abf1-4626c72d3ce7-kube-api-access-gccrk\") pod \"sub-test-model-919cc9-kserve-c9fcd7787-g4hv2\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.298418 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.298335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:42.573380 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.573349    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"]
Sep 14 08:16:42.574764 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:16:42.574728    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod261b5e67_3607_4ea9_abf1_4626c72d3ce7.slice/crio-789ecc0ced05e5944ddf581cdda08105f95cb0710e61eb59848dfcb29c6f72de WatchSource:0}: Error finding container 789ecc0ced05e5944ddf581cdda08105f95cb0710e61eb59848dfcb29c6f72de: Status 404 returned error can't find the container with id 789ecc0ced05e5944ddf581cdda08105f95cb0710e61eb59848dfcb29c6f72de
Sep 14 08:16:42.780722 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.780680    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" event={"ID":"261b5e67-3607-4ea9-abf1-4626c72d3ce7","Type":"ContainerStarted","Data":"ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065"}
Sep 14 08:16:42.780722 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.780728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" event={"ID":"261b5e67-3607-4ea9-abf1-4626c72d3ce7","Type":"ContainerStarted","Data":"789ecc0ced05e5944ddf581cdda08105f95cb0710e61eb59848dfcb29c6f72de"}
Sep 14 08:16:42.798504 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:42.798457    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" podStartSLOduration=1.7984429039999998 podStartE2EDuration="1.798442904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:42.796332189 +0000 UTC m=+1659.295724967" watchObservedRunningTime="2026-09-14 08:16:42.798442904 +0000 UTC m=+1659.297835681"
Sep 14 08:16:52.299403 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:52.299359    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:52.299774 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:52.299415    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:52.312032 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:52.312004    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:16:52.854304 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:16:52.854272    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:17:05.283590 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.283555    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"]
Sep 14 08:17:05.287514 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.287493    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.290538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.290514    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"kube-root-ca.crt\""
Sep 14 08:17:05.290667 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.290575    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"sub-test-model-fc0d6e-kserve-self-signed-certs\""
Sep 14 08:17:05.290667 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.290591    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"default-dockercfg-w8zwm\""
Sep 14 08:17:05.290667 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.290595    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"openshift-service-ca.crt\""
Sep 14 08:17:05.298183 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.298159    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"]
Sep 14 08:17:05.362086 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.362054    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tls-certs\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.362237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.362093    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tmp-dir\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.362237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.362122    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-dshm\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.362237 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.362145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-model-cache\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.362346 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.362260    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-home\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.362346 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.362289    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz96t\" (UniqueName: \"kubernetes.io/projected/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-kube-api-access-cz96t\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463269 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463224    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-home\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463269 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463275    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz96t\" (UniqueName: \"kubernetes.io/projected/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-kube-api-access-cz96t\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463312    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tls-certs\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tmp-dir\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463356    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-dshm\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463489    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-model-cache\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463717 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-home\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463773 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463714    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tmp-dir\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.463848 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.463829    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-model-cache\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.465638 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.465615    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-dshm\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.465828 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.465809    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tls-certs\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.471265 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.471243    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz96t\" (UniqueName: \"kubernetes.io/projected/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-kube-api-access-cz96t\") pod \"sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.597380 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.597284    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:05.868054 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.868021    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"]
Sep 14 08:17:05.869538 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:17:05.869511    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f71e88f_732e_4b6e_a8f1_fb43918c42a5.slice/crio-e45faf4e97dd3380ba79693829ee413e8178a3a5207eb131306605d88d77ac66 WatchSource:0}: Error finding container e45faf4e97dd3380ba79693829ee413e8178a3a5207eb131306605d88d77ac66: Status 404 returned error can't find the container with id e45faf4e97dd3380ba79693829ee413e8178a3a5207eb131306605d88d77ac66
Sep 14 08:17:05.928105 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:05.928076    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr" event={"ID":"0f71e88f-732e-4b6e-a8f1-fb43918c42a5","Type":"ContainerStarted","Data":"e45faf4e97dd3380ba79693829ee413e8178a3a5207eb131306605d88d77ac66"}
Sep 14 08:17:06.934636 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:06.934597    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr" event={"ID":"0f71e88f-732e-4b6e-a8f1-fb43918c42a5","Type":"ContainerStarted","Data":"6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7"}
Sep 14 08:17:06.951259 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:06.951211    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr" podStartSLOduration=1.951193585 podStartE2EDuration="1.951193585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:17:06.948757582 +0000 UTC m=+1683.448150384" watchObservedRunningTime="2026-09-14 08:17:06.951193585 +0000 UTC m=+1683.450586374"
Sep 14 08:17:15.597566 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:15.597504    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:15.597566 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:15.597583    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:15.610184 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:15.610158    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:17:15.998056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:17:15.998029    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:18:05.053704 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:05.053667    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"]
Sep 14 08:18:05.106692 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:05.106658    2556 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-self-signed-certs: secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:05.106850 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:05.106732    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs podName:261b5e67-3607-4ea9-abf1-4626c72d3ce7 nodeName:}" failed. No retries permitted until 2026-09-14 08:18:05.606714898 +0000 UTC m=+1742.106107653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs") pod "sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7") : secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:05.611702 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:05.611671    2556 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-self-signed-certs: secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:05.611869 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:05.611732    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs podName:261b5e67-3607-4ea9-abf1-4626c72d3ce7 nodeName:}" failed. No retries permitted until 2026-09-14 08:18:06.611720049 +0000 UTC m=+1743.111112819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs") pod "sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7") : secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:05.860991 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:05.860939    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"]
Sep 14 08:18:06.621570 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:06.621531    2556 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-self-signed-certs: secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:06.621950 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:06.621605    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs podName:261b5e67-3607-4ea9-abf1-4626c72d3ce7 nodeName:}" failed. No retries permitted until 2026-09-14 08:18:08.62158996 +0000 UTC m=+1745.120982715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs") pod "sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7") : secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:08.642638 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:08.642600    2556 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-self-signed-certs: secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:08.643051 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:08.642699    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs podName:261b5e67-3607-4ea9-abf1-4626c72d3ce7 nodeName:}" failed. No retries permitted until 2026-09-14 08:18:12.642685338 +0000 UTC m=+1749.142078094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs") pod "sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7") : secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:09.081167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.081135    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"]
Sep 14 08:18:09.084559 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.084535    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.086727 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.086702    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-919cc9\"/\"rate-test-model-919cc9-kserve-self-signed-certs\""
Sep 14 08:18:09.091626 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.091598    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"]
Sep 14 08:18:09.147557 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.147515    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsmft\" (UniqueName: \"kubernetes.io/projected/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-kube-api-access-dsmft\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.147756 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.147568    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-home\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.147756 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.147622    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tls-certs\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.147882 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.147804    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tmp-dir\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.147882 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.147852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-dshm\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.148061 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.147952    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-model-cache\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.248601 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.248561    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tmp-dir\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.248601 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.248607    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-dshm\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.248846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.248640    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-model-cache\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.248846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.248679    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsmft\" (UniqueName: \"kubernetes.io/projected/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-kube-api-access-dsmft\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.248846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.248706    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-home\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.248846 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.248729    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tls-certs\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.249101 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.249062    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tmp-dir\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.249157 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.249126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-model-cache\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.249207 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.249130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-home\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.251000 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.250956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-dshm\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.251285 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.251268    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tls-certs\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.255551 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.255532    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsmft\" (UniqueName: \"kubernetes.io/projected/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-kube-api-access-dsmft\") pod \"rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") " pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.395411 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.395322    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:09.669105 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:09.669078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"]
Sep 14 08:18:09.671248 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:18:09.671223    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40a2d8b5_73fa_4f9b_a96c_a947d72b6ef1.slice/crio-3ec59f9934c846fbbb44fc1d3e71ec9cb7dd9f41f8c4564ed95916f33c6cb7a2 WatchSource:0}: Error finding container 3ec59f9934c846fbbb44fc1d3e71ec9cb7dd9f41f8c4564ed95916f33c6cb7a2: Status 404 returned error can't find the container with id 3ec59f9934c846fbbb44fc1d3e71ec9cb7dd9f41f8c4564ed95916f33c6cb7a2
Sep 14 08:18:10.287301 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:10.287263    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h" event={"ID":"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1","Type":"ContainerStarted","Data":"8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe"}
Sep 14 08:18:10.287301 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:10.287304    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h" event={"ID":"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1","Type":"ContainerStarted","Data":"3ec59f9934c846fbbb44fc1d3e71ec9cb7dd9f41f8c4564ed95916f33c6cb7a2"}
Sep 14 08:18:10.305056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:10.304998    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h" podStartSLOduration=1.304952154 podStartE2EDuration="1.304952154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:10.304730811 +0000 UTC m=+1746.804123601" watchObservedRunningTime="2026-09-14 08:18:10.304952154 +0000 UTC m=+1746.804344933"
Sep 14 08:18:12.682436 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:12.682403    2556 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-self-signed-certs: secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:12.682809 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:12.682476    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs podName:261b5e67-3607-4ea9-abf1-4626c72d3ce7 nodeName:}" failed. No retries permitted until 2026-09-14 08:18:20.682462081 +0000 UTC m=+1757.181854838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs") pod "sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7") : secret "sub-test-model-919cc9-kserve-self-signed-certs" not found
Sep 14 08:18:19.395949 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:19.395904    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:19.396470 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:19.395981    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:19.408761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:19.408735    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:20.272949 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.272909    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" podUID="261b5e67-3607-4ea9-abf1-4626c72d3ce7" containerName="main" containerID="cri-o://ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065" gracePeriod=45
Sep 14 08:18:20.352617 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.352591    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:18:20.527302 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.527244    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:18:20.658552 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658505    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs\") pod \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") "
Sep 14 08:18:20.658732 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658579    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-home\") pod \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") "
Sep 14 08:18:20.658732 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658670    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-model-cache\") pod \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") "
Sep 14 08:18:20.658732 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658707    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tmp-dir\") pod \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") "
Sep 14 08:18:20.659021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658754    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-dshm\") pod \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") "
Sep 14 08:18:20.659021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658791    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gccrk\" (UniqueName: \"kubernetes.io/projected/261b5e67-3607-4ea9-abf1-4626c72d3ce7-kube-api-access-gccrk\") pod \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\" (UID: \"261b5e67-3607-4ea9-abf1-4626c72d3ce7\") "
Sep 14 08:18:20.659021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658794    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-home" (OuterVolumeSpecName: "home") pod "261b5e67-3607-4ea9-abf1-4626c72d3ce7" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:20.659021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.658987    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "261b5e67-3607-4ea9-abf1-4626c72d3ce7" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:20.659021 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.659002    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-model-cache" (OuterVolumeSpecName: "model-cache") pod "261b5e67-3607-4ea9-abf1-4626c72d3ce7" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:20.659277 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.659184    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:20.659277 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.659204    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:20.659277 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.659218    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:20.660786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.660767    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-dshm" (OuterVolumeSpecName: "dshm") pod "261b5e67-3607-4ea9-abf1-4626c72d3ce7" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:20.660849 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.660798    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "261b5e67-3607-4ea9-abf1-4626c72d3ce7" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:18:20.660907 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.660889    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261b5e67-3607-4ea9-abf1-4626c72d3ce7-kube-api-access-gccrk" (OuterVolumeSpecName: "kube-api-access-gccrk") pod "261b5e67-3607-4ea9-abf1-4626c72d3ce7" (UID: "261b5e67-3607-4ea9-abf1-4626c72d3ce7"). InnerVolumeSpecName "kube-api-access-gccrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:20.760127 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.760084    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/261b5e67-3607-4ea9-abf1-4626c72d3ce7-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:20.760127 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.760124    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gccrk\" (UniqueName: \"kubernetes.io/projected/261b5e67-3607-4ea9-abf1-4626c72d3ce7-kube-api-access-gccrk\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:20.760127 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.760136    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/261b5e67-3607-4ea9-abf1-4626c72d3ce7-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:20.879877 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:20.879790    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr" podUID="0f71e88f-732e-4b6e-a8f1-fb43918c42a5" containerName="main" containerID="cri-o://6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7" gracePeriod=45
Sep 14 08:18:21.127860 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.127832    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:18:21.265894 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.265858    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz96t\" (UniqueName: \"kubernetes.io/projected/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-kube-api-access-cz96t\") pod \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") "
Sep 14 08:18:21.266099 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.265898    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tls-certs\") pod \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") "
Sep 14 08:18:21.266099 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.265917    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tmp-dir\") pod \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") "
Sep 14 08:18:21.266099 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.265989    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-dshm\") pod \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") "
Sep 14 08:18:21.266099 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266080    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-home\") pod \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") "
Sep 14 08:18:21.266334 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266131    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-model-cache\") pod \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\" (UID: \"0f71e88f-732e-4b6e-a8f1-fb43918c42a5\") "
Sep 14 08:18:21.266334 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266225    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0f71e88f-732e-4b6e-a8f1-fb43918c42a5" (UID: "0f71e88f-732e-4b6e-a8f1-fb43918c42a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:21.266427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266347    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-home" (OuterVolumeSpecName: "home") pod "0f71e88f-732e-4b6e-a8f1-fb43918c42a5" (UID: "0f71e88f-732e-4b6e-a8f1-fb43918c42a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:21.266427 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266414    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-model-cache" (OuterVolumeSpecName: "model-cache") pod "0f71e88f-732e-4b6e-a8f1-fb43918c42a5" (UID: "0f71e88f-732e-4b6e-a8f1-fb43918c42a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:21.266523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266508    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:21.266572 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266522    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:21.266572 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.266531    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:21.267936 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.267915    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-kube-api-access-cz96t" (OuterVolumeSpecName: "kube-api-access-cz96t") pod "0f71e88f-732e-4b6e-a8f1-fb43918c42a5" (UID: "0f71e88f-732e-4b6e-a8f1-fb43918c42a5"). InnerVolumeSpecName "kube-api-access-cz96t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:21.268356 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.268326    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-dshm" (OuterVolumeSpecName: "dshm") pod "0f71e88f-732e-4b6e-a8f1-fb43918c42a5" (UID: "0f71e88f-732e-4b6e-a8f1-fb43918c42a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:18:21.268454 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.268326    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0f71e88f-732e-4b6e-a8f1-fb43918c42a5" (UID: "0f71e88f-732e-4b6e-a8f1-fb43918c42a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:18:21.348293 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.348258    2556 generic.go:356] "Generic (PLEG): container finished" podID="261b5e67-3607-4ea9-abf1-4626c72d3ce7" containerID="ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065" exitCode=0
Sep 14 08:18:21.348476 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.348316    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"
Sep 14 08:18:21.348476 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.348355    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" event={"ID":"261b5e67-3607-4ea9-abf1-4626c72d3ce7","Type":"ContainerDied","Data":"ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065"}
Sep 14 08:18:21.348476 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.348399    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2" event={"ID":"261b5e67-3607-4ea9-abf1-4626c72d3ce7","Type":"ContainerDied","Data":"789ecc0ced05e5944ddf581cdda08105f95cb0710e61eb59848dfcb29c6f72de"}
Sep 14 08:18:21.348476 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.348416    2556 scope.go:117] "RemoveContainer" containerID="ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065"
Sep 14 08:18:21.349902 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.349879    2556 generic.go:356] "Generic (PLEG): container finished" podID="0f71e88f-732e-4b6e-a8f1-fb43918c42a5" containerID="6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7" exitCode=0
Sep 14 08:18:21.350063 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.349950    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"
Sep 14 08:18:21.350063 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.349984    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr" event={"ID":"0f71e88f-732e-4b6e-a8f1-fb43918c42a5","Type":"ContainerDied","Data":"6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7"}
Sep 14 08:18:21.350063 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.350029    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr" event={"ID":"0f71e88f-732e-4b6e-a8f1-fb43918c42a5","Type":"ContainerDied","Data":"e45faf4e97dd3380ba79693829ee413e8178a3a5207eb131306605d88d77ac66"}
Sep 14 08:18:21.358567 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.358545    2556 scope.go:117] "RemoveContainer" containerID="ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065"
Sep 14 08:18:21.358831 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:21.358811    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065\": container with ID starting with ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065 not found: ID does not exist" containerID="ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065"
Sep 14 08:18:21.358908 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.358842    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065"} err="failed to get container status \"ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065\": rpc error: code = NotFound desc = could not find container \"ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065\": container with ID starting with ba8b6a513564e828f622e082a671b1e8133ab0b7face86bdada132e09a7e5065 not found: ID does not exist"
Sep 14 08:18:21.358908 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.358866    2556 scope.go:117] "RemoveContainer" containerID="6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7"
Sep 14 08:18:21.367402 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.367380    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz96t\" (UniqueName: \"kubernetes.io/projected/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-kube-api-access-cz96t\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:21.367486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.367404    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:21.367486 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.367418    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f71e88f-732e-4b6e-a8f1-fb43918c42a5-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:18:21.367798 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.367777    2556 scope.go:117] "RemoveContainer" containerID="6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7"
Sep 14 08:18:21.368063 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:18:21.368045    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7\": container with ID starting with 6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7 not found: ID does not exist" containerID="6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7"
Sep 14 08:18:21.368124 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.368071    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7"} err="failed to get container status \"6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7\": rpc error: code = NotFound desc = could not find container \"6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7\": container with ID starting with 6cd1cb54673f0ecb596231b9d0160558bdcf43d073fe80ad6cf5ba06402bbbf7 not found: ID does not exist"
Sep 14 08:18:21.373335 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.373316    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"]
Sep 14 08:18:21.375541 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.375521    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/sub-test-model-919cc9-kserve-c9fcd7787-g4hv2"]
Sep 14 08:18:21.382864 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.382834    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"]
Sep 14 08:18:21.384870 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:21.384839    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/sub-test-model-fc0d6e-kserve-79c8cdf48c-pqcdr"]
Sep 14 08:18:22.019414 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:22.019376    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f71e88f-732e-4b6e-a8f1-fb43918c42a5" path="/var/lib/kubelet/pods/0f71e88f-732e-4b6e-a8f1-fb43918c42a5/volumes"
Sep 14 08:18:22.019777 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:22.019768    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261b5e67-3607-4ea9-abf1-4626c72d3ce7" path="/var/lib/kubelet/pods/261b5e67-3607-4ea9-abf1-4626c72d3ce7/volumes"
Sep 14 08:18:39.682469 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.682430    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"]
Sep 14 08:18:39.683122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.682926    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f71e88f-732e-4b6e-a8f1-fb43918c42a5" containerName="main"
Sep 14 08:18:39.683122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.682946    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f71e88f-732e-4b6e-a8f1-fb43918c42a5" containerName="main"
Sep 14 08:18:39.683122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.682989    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261b5e67-3607-4ea9-abf1-4626c72d3ce7" containerName="main"
Sep 14 08:18:39.683122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.683000    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="261b5e67-3607-4ea9-abf1-4626c72d3ce7" containerName="main"
Sep 14 08:18:39.683122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.683110    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="261b5e67-3607-4ea9-abf1-4626c72d3ce7" containerName="main"
Sep 14 08:18:39.683413 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.683151    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f71e88f-732e-4b6e-a8f1-fb43918c42a5" containerName="main"
Sep 14 08:18:39.686224 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.686207    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.688987 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.688944    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"rate-test-model-fc0d6e-kserve-self-signed-certs\""
Sep 14 08:18:39.688987 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.688979    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"openshift-service-ca.crt\""
Sep 14 08:18:39.689194 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.689027    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"kube-root-ca.crt\""
Sep 14 08:18:39.689194 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.689032    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc0d6e\"/\"default-dockercfg-w8zwm\""
Sep 14 08:18:39.695347 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.695325    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"]
Sep 14 08:18:39.720402 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.720371    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-model-cache\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.720543 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.720410    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p74vt\" (UniqueName: \"kubernetes.io/projected/9e0281cf-6dee-4717-b264-7a5507b48792-kube-api-access-p74vt\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.720543 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.720494    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-dshm\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.720640 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.720569    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-tmp-dir\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.720640 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.720609    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-home\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.720713 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.720655    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e0281cf-6dee-4717-b264-7a5507b48792-tls-certs\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821386 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-dshm\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821576 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-tmp-dir\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821576 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-home\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821576 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821461    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e0281cf-6dee-4717-b264-7a5507b48792-tls-certs\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821576 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821492    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-model-cache\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821576 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p74vt\" (UniqueName: \"kubernetes.io/projected/9e0281cf-6dee-4717-b264-7a5507b48792-kube-api-access-p74vt\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821882 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821854    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-tmp-dir\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821933 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821908    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-home\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.821933 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.821908    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-model-cache\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.823592 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.823568    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-dshm\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.823863 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.823844    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e0281cf-6dee-4717-b264-7a5507b48792-tls-certs\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.830814 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.830793    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p74vt\" (UniqueName: \"kubernetes.io/projected/9e0281cf-6dee-4717-b264-7a5507b48792-kube-api-access-p74vt\") pod \"rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") " pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:39.998547 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:39.998467    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:40.271048 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:40.271018    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"]
Sep 14 08:18:40.272625 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:18:40.272600    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e0281cf_6dee_4717_b264_7a5507b48792.slice/crio-a198bf8d2f5a74add4e3293044ecb410daa5140a6d233f85a26387cee1257c1d WatchSource:0}: Error finding container a198bf8d2f5a74add4e3293044ecb410daa5140a6d233f85a26387cee1257c1d: Status 404 returned error can't find the container with id a198bf8d2f5a74add4e3293044ecb410daa5140a6d233f85a26387cee1257c1d
Sep 14 08:18:40.458374 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:40.458333    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl" event={"ID":"9e0281cf-6dee-4717-b264-7a5507b48792","Type":"ContainerStarted","Data":"613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b"}
Sep 14 08:18:40.458374 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:40.458375    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl" event={"ID":"9e0281cf-6dee-4717-b264-7a5507b48792","Type":"ContainerStarted","Data":"a198bf8d2f5a74add4e3293044ecb410daa5140a6d233f85a26387cee1257c1d"}
Sep 14 08:18:40.475759 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:40.475701    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl" podStartSLOduration=1.475684465 podStartE2EDuration="1.475684465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:40.472812198 +0000 UTC m=+1776.972204977" watchObservedRunningTime="2026-09-14 08:18:40.475684465 +0000 UTC m=+1776.975077243"
Sep 14 08:18:49.999186 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:49.999128    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:49.999186 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:49.999193    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:50.012027 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:50.012003    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:18:50.527138 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:18:50.527104    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:19:19.807681 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:19.807645    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"]
Sep 14 08:19:20.796369 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:20.796335    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"]
Sep 14 08:19:27.286948 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.286907    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"]
Sep 14 08:19:27.290166 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.290145    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.292346 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.292327    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-03d6fa-dockercfg-qkqn2\""
Sep 14 08:19:27.292450 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.292348    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-03d6fa\""
Sep 14 08:19:27.299160 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.299136    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"]
Sep 14 08:19:27.320189 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.320141    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"]
Sep 14 08:19:27.324328 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.324307    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.332493 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.332464    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"]
Sep 14 08:19:27.340941 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.340913    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cksfn\" (UniqueName: \"kubernetes.io/projected/4aa58e4e-47ba-478f-9223-ae269e303f4b-kube-api-access-cksfn\") pod \"payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.341122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.341044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461dd94-363f-45bb-945f-842cadbf52cd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.341122 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.341107    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc5wx\" (UniqueName: \"kubernetes.io/projected/5461dd94-363f-45bb-945f-842cadbf52cd-kube-api-access-sc5wx\") pod \"payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.341236 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.341145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa58e4e-47ba-478f-9223-ae269e303f4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.442594 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.442552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cksfn\" (UniqueName: \"kubernetes.io/projected/4aa58e4e-47ba-478f-9223-ae269e303f4b-kube-api-access-cksfn\") pod \"payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.442786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.442617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461dd94-363f-45bb-945f-842cadbf52cd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.442786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.442655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc5wx\" (UniqueName: \"kubernetes.io/projected/5461dd94-363f-45bb-945f-842cadbf52cd-kube-api-access-sc5wx\") pod \"payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.442786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.442679    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa58e4e-47ba-478f-9223-ae269e303f4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.443410 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.443388    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa58e4e-47ba-478f-9223-ae269e303f4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.443455 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.443390    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461dd94-363f-45bb-945f-842cadbf52cd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.450089 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.450060    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cksfn\" (UniqueName: \"kubernetes.io/projected/4aa58e4e-47ba-478f-9223-ae269e303f4b-kube-api-access-cksfn\") pod \"payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.450216 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.450071    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc5wx\" (UniqueName: \"kubernetes.io/projected/5461dd94-363f-45bb-945f-842cadbf52cd-kube-api-access-sc5wx\") pod \"payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.601202 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.601129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:27.637167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.637137    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:27.880399 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.880370    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"]
Sep 14 08:19:27.880740 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:19:27.880713    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa58e4e_47ba_478f_9223_ae269e303f4b.slice/crio-5bb02354b8b735a3e70b57b4e48899d022d6552157383a0e314e327421b0172d WatchSource:0}: Error finding container 5bb02354b8b735a3e70b57b4e48899d022d6552157383a0e314e327421b0172d: Status 404 returned error can't find the container with id 5bb02354b8b735a3e70b57b4e48899d022d6552157383a0e314e327421b0172d
Sep 14 08:19:27.917250 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:27.917223    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"]
Sep 14 08:19:27.919620 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:19:27.919599    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5461dd94_363f_45bb_945f_842cadbf52cd.slice/crio-33e96733719522c02c0aac736832c920b62939498aeaaa9a5d6da47f4aca50c5 WatchSource:0}: Error finding container 33e96733719522c02c0aac736832c920b62939498aeaaa9a5d6da47f4aca50c5: Status 404 returned error can't find the container with id 33e96733719522c02c0aac736832c920b62939498aeaaa9a5d6da47f4aca50c5
Sep 14 08:19:28.738382 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.738338    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn" event={"ID":"4aa58e4e-47ba-478f-9223-ae269e303f4b","Type":"ContainerStarted","Data":"6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66"}
Sep 14 08:19:28.738835 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.738414    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn" event={"ID":"4aa58e4e-47ba-478f-9223-ae269e303f4b","Type":"ContainerStarted","Data":"5bb02354b8b735a3e70b57b4e48899d022d6552157383a0e314e327421b0172d"}
Sep 14 08:19:28.738835 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.738533    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:28.740017 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.739995    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47" event={"ID":"5461dd94-363f-45bb-945f-842cadbf52cd","Type":"ContainerStarted","Data":"d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8"}
Sep 14 08:19:28.740100 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.740037    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:28.740100 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.740058    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47" event={"ID":"5461dd94-363f-45bb-945f-842cadbf52cd","Type":"ContainerStarted","Data":"33e96733719522c02c0aac736832c920b62939498aeaaa9a5d6da47f4aca50c5"}
Sep 14 08:19:28.756785 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.756732    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn" podStartSLOduration=1.756713966 podStartE2EDuration="1.756713966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:28.753229911 +0000 UTC m=+1825.252622694" watchObservedRunningTime="2026-09-14 08:19:28.756713966 +0000 UTC m=+1825.256106744"
Sep 14 08:19:28.769766 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:28.769717    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47" podStartSLOduration=1.7697015029999998 podStartE2EDuration="1.769701503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:28.767146038 +0000 UTC m=+1825.266538833" watchObservedRunningTime="2026-09-14 08:19:28.769701503 +0000 UTC m=+1825.269094315"
Sep 14 08:19:34.827839 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:34.827798    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h" podUID="40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" containerName="main" containerID="cri-o://8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe" gracePeriod=45
Sep 14 08:19:35.089288 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.089224    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:19:35.115558 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.115518    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsmft\" (UniqueName: \"kubernetes.io/projected/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-kube-api-access-dsmft\") pod \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") "
Sep 14 08:19:35.115761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.115582    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tls-certs\") pod \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") "
Sep 14 08:19:35.115761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.115650    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-home\") pod \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") "
Sep 14 08:19:35.115761 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.115722    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-dshm\") pod \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") "
Sep 14 08:19:35.115921 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.115793    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tmp-dir\") pod \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") "
Sep 14 08:19:35.115921 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.115824    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-model-cache\") pod \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\" (UID: \"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1\") "
Sep 14 08:19:35.116352 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.116320    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-home" (OuterVolumeSpecName: "home") pod "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" (UID: "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:35.116610 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.116573    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-model-cache" (OuterVolumeSpecName: "model-cache") pod "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" (UID: "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:35.117521 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.117234    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" (UID: "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:35.118894 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.118862    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-dshm" (OuterVolumeSpecName: "dshm") pod "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" (UID: "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:35.119171 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.119145    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" (UID: "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:19:35.119296 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.119275    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-kube-api-access-dsmft" (OuterVolumeSpecName: "kube-api-access-dsmft") pod "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" (UID: "40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1"). InnerVolumeSpecName "kube-api-access-dsmft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:35.217762 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.217722    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:35.217762 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.217756    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:35.217762 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.217766    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsmft\" (UniqueName: \"kubernetes.io/projected/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-kube-api-access-dsmft\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:35.218017 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.217775    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:35.218017 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.217785    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:35.218017 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.217792    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:35.787228 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.787190    2556 generic.go:356] "Generic (PLEG): container finished" podID="40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" containerID="8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe" exitCode=0
Sep 14 08:19:35.787388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.787254    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"
Sep 14 08:19:35.787388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.787276    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h" event={"ID":"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1","Type":"ContainerDied","Data":"8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe"}
Sep 14 08:19:35.787388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.787324    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h" event={"ID":"40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1","Type":"ContainerDied","Data":"3ec59f9934c846fbbb44fc1d3e71ec9cb7dd9f41f8c4564ed95916f33c6cb7a2"}
Sep 14 08:19:35.787388 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.787343    2556 scope.go:117] "RemoveContainer" containerID="8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe"
Sep 14 08:19:35.796887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.796869    2556 scope.go:117] "RemoveContainer" containerID="8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe"
Sep 14 08:19:35.797143 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:19:35.797125    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe\": container with ID starting with 8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe not found: ID does not exist" containerID="8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe"
Sep 14 08:19:35.797201 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.797151    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe"} err="failed to get container status \"8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe\": rpc error: code = NotFound desc = could not find container \"8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe\": container with ID starting with 8970bc8107a6b14a06a097f09c2313bc314374711dafa514092eb698d5ceb4fe not found: ID does not exist"
Sep 14 08:19:35.807057 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.807034    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"]
Sep 14 08:19:35.812339 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.812318    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-919cc9/rate-test-model-919cc9-kserve-6c546d4cbc-7bq8h"]
Sep 14 08:19:35.815407 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:35.815385    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl" podUID="9e0281cf-6dee-4717-b264-7a5507b48792" containerName="main" containerID="cri-o://613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b" gracePeriod=45
Sep 14 08:19:36.019529 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.019498    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" path="/var/lib/kubelet/pods/40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1/volumes"
Sep 14 08:19:36.073013 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.072986    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:19:36.125523 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125491    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-home\") pod \"9e0281cf-6dee-4717-b264-7a5507b48792\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") "
Sep 14 08:19:36.125703 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125566    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-dshm\") pod \"9e0281cf-6dee-4717-b264-7a5507b48792\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") "
Sep 14 08:19:36.125703 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125603    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p74vt\" (UniqueName: \"kubernetes.io/projected/9e0281cf-6dee-4717-b264-7a5507b48792-kube-api-access-p74vt\") pod \"9e0281cf-6dee-4717-b264-7a5507b48792\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") "
Sep 14 08:19:36.125703 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125629    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-tmp-dir\") pod \"9e0281cf-6dee-4717-b264-7a5507b48792\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") "
Sep 14 08:19:36.125703 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125663    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-model-cache\") pod \"9e0281cf-6dee-4717-b264-7a5507b48792\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") "
Sep 14 08:19:36.125703 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125702    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e0281cf-6dee-4717-b264-7a5507b48792-tls-certs\") pod \"9e0281cf-6dee-4717-b264-7a5507b48792\" (UID: \"9e0281cf-6dee-4717-b264-7a5507b48792\") "
Sep 14 08:19:36.125948 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125786    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-home" (OuterVolumeSpecName: "home") pod "9e0281cf-6dee-4717-b264-7a5507b48792" (UID: "9e0281cf-6dee-4717-b264-7a5507b48792"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:36.125948 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125895    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9e0281cf-6dee-4717-b264-7a5507b48792" (UID: "9e0281cf-6dee-4717-b264-7a5507b48792"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:36.125948 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.125927    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-model-cache" (OuterVolumeSpecName: "model-cache") pod "9e0281cf-6dee-4717-b264-7a5507b48792" (UID: "9e0281cf-6dee-4717-b264-7a5507b48792"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:36.126090 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.126023    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:36.126090 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.126042    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:36.126090 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.126055    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:36.127808 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.127778    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-dshm" (OuterVolumeSpecName: "dshm") pod "9e0281cf-6dee-4717-b264-7a5507b48792" (UID: "9e0281cf-6dee-4717-b264-7a5507b48792"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:36.127808 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.127784    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e0281cf-6dee-4717-b264-7a5507b48792-kube-api-access-p74vt" (OuterVolumeSpecName: "kube-api-access-p74vt") pod "9e0281cf-6dee-4717-b264-7a5507b48792" (UID: "9e0281cf-6dee-4717-b264-7a5507b48792"). InnerVolumeSpecName "kube-api-access-p74vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:36.127992 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.127808    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e0281cf-6dee-4717-b264-7a5507b48792-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9e0281cf-6dee-4717-b264-7a5507b48792" (UID: "9e0281cf-6dee-4717-b264-7a5507b48792"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:19:36.226662 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.226623    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e0281cf-6dee-4717-b264-7a5507b48792-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:36.226662 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.226659    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p74vt\" (UniqueName: \"kubernetes.io/projected/9e0281cf-6dee-4717-b264-7a5507b48792-kube-api-access-p74vt\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:36.226869 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.226674    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e0281cf-6dee-4717-b264-7a5507b48792-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:19:36.794629 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.794590    2556 generic.go:356] "Generic (PLEG): container finished" podID="9e0281cf-6dee-4717-b264-7a5507b48792" containerID="613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b" exitCode=0
Sep 14 08:19:36.794806 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.794637    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"
Sep 14 08:19:36.794806 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.794680    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl" event={"ID":"9e0281cf-6dee-4717-b264-7a5507b48792","Type":"ContainerDied","Data":"613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b"}
Sep 14 08:19:36.794806 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.794728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl" event={"ID":"9e0281cf-6dee-4717-b264-7a5507b48792","Type":"ContainerDied","Data":"a198bf8d2f5a74add4e3293044ecb410daa5140a6d233f85a26387cee1257c1d"}
Sep 14 08:19:36.794806 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.794750    2556 scope.go:117] "RemoveContainer" containerID="613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b"
Sep 14 08:19:36.803762 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.803745    2556 scope.go:117] "RemoveContainer" containerID="613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b"
Sep 14 08:19:36.804028 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:19:36.804009    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b\": container with ID starting with 613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b not found: ID does not exist" containerID="613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b"
Sep 14 08:19:36.804078 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.804037    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b"} err="failed to get container status \"613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b\": rpc error: code = NotFound desc = could not find container \"613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b\": container with ID starting with 613b2a9b1687e8c6f24d7aecbeb372203a8b717715871028166f5a99d120996b not found: ID does not exist"
Sep 14 08:19:36.814660 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.814639    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"]
Sep 14 08:19:36.818167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:36.818147    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc0d6e/rate-test-model-fc0d6e-kserve-6f667468c5-bc6rl"]
Sep 14 08:19:38.018857 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:38.018819    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e0281cf-6dee-4717-b264-7a5507b48792" path="/var/lib/kubelet/pods/9e0281cf-6dee-4717-b264-7a5507b48792/volumes"
Sep 14 08:19:39.752933 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:39.752903    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:19:39.753332 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:39.752985    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:19:55.625546 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.625508    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"]
Sep 14 08:19:55.626177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.626155    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" containerName="main"
Sep 14 08:19:55.626233 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.626182    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" containerName="main"
Sep 14 08:19:55.626233 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.626195    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0281cf-6dee-4717-b264-7a5507b48792" containerName="main"
Sep 14 08:19:55.626233 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.626204    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0281cf-6dee-4717-b264-7a5507b48792" containerName="main"
Sep 14 08:19:55.626338 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.626319    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0281cf-6dee-4717-b264-7a5507b48792" containerName="main"
Sep 14 08:19:55.626338 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.626331    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a2d8b5-73fa-4f9b-a96c-a947d72b6ef1" containerName="main"
Sep 14 08:19:55.631193 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.631166    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.633349 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.633322    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-7bcf7b-serving-cert\""
Sep 14 08:19:55.640067 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.640038    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"]
Sep 14 08:19:55.802779 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.802738    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869596d6-ab63-4615-8c1f-8f2853363220-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.802779 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.802777    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-metrics-certs\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.803017 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.802887    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.803017 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.802916    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfnhm\" (UniqueName: \"kubernetes.io/projected/869596d6-ab63-4615-8c1f-8f2853363220-kube-api-access-hfnhm\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.903755 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.903664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.903755 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.903704    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfnhm\" (UniqueName: \"kubernetes.io/projected/869596d6-ab63-4615-8c1f-8f2853363220-kube-api-access-hfnhm\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.904072 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.903773    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869596d6-ab63-4615-8c1f-8f2853363220-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.904072 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.903799    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-metrics-certs\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.904444 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.904420    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869596d6-ab63-4615-8c1f-8f2853363220-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.906258 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.906235    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-metrics-certs\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.906368 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.906285    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.911701 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.911672    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfnhm\" (UniqueName: \"kubernetes.io/projected/869596d6-ab63-4615-8c1f-8f2853363220-kube-api-access-hfnhm\") pod \"maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:55.945244 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:55.945207    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:56.254477 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.254448    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"]
Sep 14 08:19:56.256328 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:19:56.256300    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869596d6_ab63_4615_8c1f_8f2853363220.slice/crio-6ec325aef753d625f0661e1cd32affa48ab4e1deb3cf83f6511ce9569dbf9c27 WatchSource:0}: Error finding container 6ec325aef753d625f0661e1cd32affa48ab4e1deb3cf83f6511ce9569dbf9c27: Status 404 returned error can't find the container with id 6ec325aef753d625f0661e1cd32affa48ab4e1deb3cf83f6511ce9569dbf9c27
Sep 14 08:19:56.257686 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.257663    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:19:56.522521 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.522441    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"]
Sep 14 08:19:56.526887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.526866    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.529272 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.529245    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7bcf7b\""
Sep 14 08:19:56.529390 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.529354    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7bcf7b-dockercfg-skcdx\""
Sep 14 08:19:56.533548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.533527    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"]
Sep 14 08:19:56.609862 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.609819    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f13268f1-24a6-414a-8d74-96649783b907-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.609862 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.609862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q25m\" (UniqueName: \"kubernetes.io/projected/f13268f1-24a6-414a-8d74-96649783b907-kube-api-access-6q25m\") pod \"payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.710728 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.710697    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f13268f1-24a6-414a-8d74-96649783b907-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.711087 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.710736    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q25m\" (UniqueName: \"kubernetes.io/projected/f13268f1-24a6-414a-8d74-96649783b907-kube-api-access-6q25m\") pod \"payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.711391 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.711371    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f13268f1-24a6-414a-8d74-96649783b907-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.718641 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.718613    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q25m\" (UniqueName: \"kubernetes.io/projected/f13268f1-24a6-414a-8d74-96649783b907-kube-api-access-6q25m\") pod \"payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.839383 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.839298    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:19:56.911287 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.911255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh" event={"ID":"869596d6-ab63-4615-8c1f-8f2853363220","Type":"ContainerStarted","Data":"9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9"}
Sep 14 08:19:56.911511 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.911427    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh" event={"ID":"869596d6-ab63-4615-8c1f-8f2853363220","Type":"ContainerStarted","Data":"6ec325aef753d625f0661e1cd32affa48ab4e1deb3cf83f6511ce9569dbf9c27"}
Sep 14 08:19:56.911841 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.911817    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:19:56.929285 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:56.929233    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh" podStartSLOduration=1.502045742 podStartE2EDuration="1.929213009s" totalImagesPullingTime="427.167267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:56.92780079 +0000 UTC m=+1853.427193595" watchObservedRunningTime="2026-09-14 08:19:56.929213009 +0000 UTC m=+1853.428605787"
Sep 14 08:19:57.110472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:57.110444    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"]
Sep 14 08:19:57.111662 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:19:57.111631    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13268f1_24a6_414a_8d74_96649783b907.slice/crio-ce53d38ea052ae5047c4475c6ffa5aad540313e58986c5a389464e78b7c18ea7 WatchSource:0}: Error finding container ce53d38ea052ae5047c4475c6ffa5aad540313e58986c5a389464e78b7c18ea7: Status 404 returned error can't find the container with id ce53d38ea052ae5047c4475c6ffa5aad540313e58986c5a389464e78b7c18ea7
Sep 14 08:19:57.919674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:57.919631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp" event={"ID":"f13268f1-24a6-414a-8d74-96649783b907","Type":"ContainerStarted","Data":"381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd"}
Sep 14 08:19:57.920103 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:57.919697    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp" event={"ID":"f13268f1-24a6-414a-8d74-96649783b907","Type":"ContainerStarted","Data":"ce53d38ea052ae5047c4475c6ffa5aad540313e58986c5a389464e78b7c18ea7"}
Sep 14 08:19:57.936082 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:57.936039    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp" podStartSLOduration=1.936023074 podStartE2EDuration="1.936023074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:57.933854756 +0000 UTC m=+1854.433247559" watchObservedRunningTime="2026-09-14 08:19:57.936023074 +0000 UTC m=+1854.435415905"
Sep 14 08:19:58.924548 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:19:58.924512    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:20:02.925651 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:20:02.925621    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:20:09.932856 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:20:09.932823    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:20:23.858725 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:20:23.858690    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q6hxd_1231c9fa-d159-43fd-8c3a-024a981babde/payload-processing/0.log"
Sep 14 08:20:23.983974 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:20:23.983938    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp_f13268f1-24a6-414a-8d74-96649783b907/payload-processing/0.log"
Sep 14 08:21:04.280478 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:04.280445    2556 scope.go:117] "RemoveContainer" containerID="9fa1e7b6c63528cc8d53c5e42b5f7dd758a886f57ad9f44edb65b753aa79e14f"
Sep 14 08:21:05.107002 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:05.106958    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47_5461dd94-363f-45bb-945f-842cadbf52cd/payload-processing/0.log"
Sep 14 08:21:05.236260 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:05.236228    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q6hxd_1231c9fa-d159-43fd-8c3a-024a981babde/payload-processing/0.log"
Sep 14 08:21:05.369374 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:05.369282    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp_f13268f1-24a6-414a-8d74-96649783b907/payload-processing/0.log"
Sep 14 08:21:12.052866 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.052830    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"]
Sep 14 08:21:12.056540 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.056517    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.058598 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.058579    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-70893f-serving-cert\""
Sep 14 08:21:12.067152 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.067131    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"]
Sep 14 08:21:12.173499 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.173460    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.173666 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.173508    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.173666 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.173581    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7l2p\" (UniqueName: \"kubernetes.io/projected/4d4698bb-1a45-456e-bfbc-6a504648fc2d-kube-api-access-z7l2p\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.173744 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.173721    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4698bb-1a45-456e-bfbc-6a504648fc2d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.274609 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.274577    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.274803 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.274631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.274803 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.274659    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7l2p\" (UniqueName: \"kubernetes.io/projected/4d4698bb-1a45-456e-bfbc-6a504648fc2d-kube-api-access-z7l2p\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.274803 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.274756    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4698bb-1a45-456e-bfbc-6a504648fc2d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.274803 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:21:12.274786    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-serving-cert: secret "maas-api-e2e-ipp-cleanup-70893f-serving-cert" not found
Sep 14 08:21:12.275067 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:21:12.274862    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls podName:4d4698bb-1a45-456e-bfbc-6a504648fc2d nodeName:}" failed. No retries permitted until 2026-09-14 08:21:12.774839935 +0000 UTC m=+1929.274232695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" (UID: "4d4698bb-1a45-456e-bfbc-6a504648fc2d") : secret "maas-api-e2e-ipp-cleanup-70893f-serving-cert" not found
Sep 14 08:21:12.275472 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.275449    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4698bb-1a45-456e-bfbc-6a504648fc2d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.276944 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.276926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.282328 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.282299    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7l2p\" (UniqueName: \"kubernetes.io/projected/4d4698bb-1a45-456e-bfbc-6a504648fc2d-kube-api-access-z7l2p\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.780214 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.780181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.782735 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.782700    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:12.968798 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:12.968760    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:13.241878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:13.241847    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"]
Sep 14 08:21:13.243609 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:21:13.243580    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4698bb_1a45_456e_bfbc_6a504648fc2d.slice/crio-86c1fd4fdfa053f33accb3f38d07e7a5b4b42ae50e10c6ac0f9d3a92d4380d16 WatchSource:0}: Error finding container 86c1fd4fdfa053f33accb3f38d07e7a5b4b42ae50e10c6ac0f9d3a92d4380d16: Status 404 returned error can't find the container with id 86c1fd4fdfa053f33accb3f38d07e7a5b4b42ae50e10c6ac0f9d3a92d4380d16
Sep 14 08:21:13.347033 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:13.346997    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" event={"ID":"4d4698bb-1a45-456e-bfbc-6a504648fc2d","Type":"ContainerStarted","Data":"86c1fd4fdfa053f33accb3f38d07e7a5b4b42ae50e10c6ac0f9d3a92d4380d16"}
Sep 14 08:21:14.354431 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:14.354385    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" event={"ID":"4d4698bb-1a45-456e-bfbc-6a504648fc2d","Type":"ContainerStarted","Data":"285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5"}
Sep 14 08:21:14.354833 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:14.354539    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:14.371333 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:14.371283    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" podStartSLOduration=1.97366477 podStartE2EDuration="2.371270885s" totalImagesPullingTime="397.606115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:21:14.369785284 +0000 UTC m=+1930.869178063" watchObservedRunningTime="2026-09-14 08:21:14.371270885 +0000 UTC m=+1930.870663662"
Sep 14 08:21:20.365926 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:20.365894    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:40.868306 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:40.868268    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"]
Sep 14 08:21:40.868841 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:40.868517    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" podUID="4d4698bb-1a45-456e-bfbc-6a504648fc2d" containerName="maas-api" containerID="cri-o://285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5" gracePeriod=30
Sep 14 08:21:41.125857 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.125797    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:41.230619 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.230587    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-metrics-certs\") pod \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") "
Sep 14 08:21:41.230791 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.230696    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4698bb-1a45-456e-bfbc-6a504648fc2d-openshift-service-ca\") pod \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") "
Sep 14 08:21:41.230791 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.230729    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7l2p\" (UniqueName: \"kubernetes.io/projected/4d4698bb-1a45-456e-bfbc-6a504648fc2d-kube-api-access-z7l2p\") pod \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") "
Sep 14 08:21:41.230791 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.230752    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls\") pod \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\" (UID: \"4d4698bb-1a45-456e-bfbc-6a504648fc2d\") "
Sep 14 08:21:41.231124 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.231094    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4698bb-1a45-456e-bfbc-6a504648fc2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d4698bb-1a45-456e-bfbc-6a504648fc2d" (UID: "4d4698bb-1a45-456e-bfbc-6a504648fc2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:41.232792 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.232761    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d4698bb-1a45-456e-bfbc-6a504648fc2d" (UID: "4d4698bb-1a45-456e-bfbc-6a504648fc2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:21:41.232891 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.232798    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d4698bb-1a45-456e-bfbc-6a504648fc2d-kube-api-access-z7l2p" (OuterVolumeSpecName: "kube-api-access-z7l2p") pod "4d4698bb-1a45-456e-bfbc-6a504648fc2d" (UID: "4d4698bb-1a45-456e-bfbc-6a504648fc2d"). InnerVolumeSpecName "kube-api-access-z7l2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:41.232891 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.232824    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4d4698bb-1a45-456e-bfbc-6a504648fc2d" (UID: "4d4698bb-1a45-456e-bfbc-6a504648fc2d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:21:41.331887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.331851    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4698bb-1a45-456e-bfbc-6a504648fc2d-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.331887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.331880    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7l2p\" (UniqueName: \"kubernetes.io/projected/4d4698bb-1a45-456e-bfbc-6a504648fc2d-kube-api-access-z7l2p\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.331887 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.331890    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.332154 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.331898    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d4698bb-1a45-456e-bfbc-6a504648fc2d-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.510227 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.510192    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d4698bb-1a45-456e-bfbc-6a504648fc2d" containerID="285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5" exitCode=0
Sep 14 08:21:41.510435 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.510231    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"
Sep 14 08:21:41.510435 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.510271    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" event={"ID":"4d4698bb-1a45-456e-bfbc-6a504648fc2d","Type":"ContainerDied","Data":"285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5"}
Sep 14 08:21:41.510435 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.510326    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r" event={"ID":"4d4698bb-1a45-456e-bfbc-6a504648fc2d","Type":"ContainerDied","Data":"86c1fd4fdfa053f33accb3f38d07e7a5b4b42ae50e10c6ac0f9d3a92d4380d16"}
Sep 14 08:21:41.510435 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.510356    2556 scope.go:117] "RemoveContainer" containerID="285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5"
Sep 14 08:21:41.520193 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.520169    2556 scope.go:117] "RemoveContainer" containerID="285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5"
Sep 14 08:21:41.520497 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:21:41.520478    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5\": container with ID starting with 285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5 not found: ID does not exist" containerID="285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5"
Sep 14 08:21:41.520558 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.520506    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5"} err="failed to get container status \"285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5\": rpc error: code = NotFound desc = could not find container \"285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5\": container with ID starting with 285492be9c1752687ae44bcd929e7a6e36a3a893e75276e61e372e01e5cbefe5 not found: ID does not exist"
Sep 14 08:21:41.530874 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.530850    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"]
Sep 14 08:21:41.534350 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:41.534327    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-70893f-585d48d7df-h979r"]
Sep 14 08:21:42.019096 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:21:42.019056    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d4698bb-1a45-456e-bfbc-6a504648fc2d" path="/var/lib/kubelet/pods/4d4698bb-1a45-456e-bfbc-6a504648fc2d/volumes"
Sep 14 08:22:04.301511 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:04.301437    2556 scope.go:117] "RemoveContainer" containerID="82465d862c670b04ff4d5a07a358e620b33b64e61118cfdb9896e43263f90bea"
Sep 14 08:22:04.309835 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:04.309810    2556 scope.go:117] "RemoveContainer" containerID="6e461bcdb6f31a8d3695fc78e33af3be922b4004aeb6c1dec9e0ed38095dd0f3"
Sep 14 08:22:06.960481 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:06.960442    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"]
Sep 14 08:22:06.960873 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:06.960673    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47" podUID="5461dd94-363f-45bb-945f-842cadbf52cd" containerName="payload-processing" containerID="cri-o://d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8" gracePeriod=30
Sep 14 08:22:06.968808 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:06.968479    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"]
Sep 14 08:22:06.968808 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:06.968738    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn" podUID="4aa58e4e-47ba-478f-9223-ae269e303f4b" containerName="payload-pre-processing" containerID="cri-o://6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66" gracePeriod=30
Sep 14 08:22:07.222822 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.222757    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:22:07.246292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.246268    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:22:07.257226 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.257201    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461dd94-363f-45bb-945f-842cadbf52cd-plugins-config-volume\") pod \"5461dd94-363f-45bb-945f-842cadbf52cd\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") "
Sep 14 08:22:07.257367 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.257295    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sc5wx\" (UniqueName: \"kubernetes.io/projected/5461dd94-363f-45bb-945f-842cadbf52cd-kube-api-access-sc5wx\") pod \"5461dd94-363f-45bb-945f-842cadbf52cd\" (UID: \"5461dd94-363f-45bb-945f-842cadbf52cd\") "
Sep 14 08:22:07.257584 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.257557    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5461dd94-363f-45bb-945f-842cadbf52cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5461dd94-363f-45bb-945f-842cadbf52cd" (UID: "5461dd94-363f-45bb-945f-842cadbf52cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:22:07.259336 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.259312    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5461dd94-363f-45bb-945f-842cadbf52cd-kube-api-access-sc5wx" (OuterVolumeSpecName: "kube-api-access-sc5wx") pod "5461dd94-363f-45bb-945f-842cadbf52cd" (UID: "5461dd94-363f-45bb-945f-842cadbf52cd"). InnerVolumeSpecName "kube-api-access-sc5wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:22:07.358797 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.358761    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cksfn\" (UniqueName: \"kubernetes.io/projected/4aa58e4e-47ba-478f-9223-ae269e303f4b-kube-api-access-cksfn\") pod \"4aa58e4e-47ba-478f-9223-ae269e303f4b\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") "
Sep 14 08:22:07.359013 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.358864    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa58e4e-47ba-478f-9223-ae269e303f4b-plugins-config-volume\") pod \"4aa58e4e-47ba-478f-9223-ae269e303f4b\" (UID: \"4aa58e4e-47ba-478f-9223-ae269e303f4b\") "
Sep 14 08:22:07.359219 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.359199    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc5wx\" (UniqueName: \"kubernetes.io/projected/5461dd94-363f-45bb-945f-842cadbf52cd-kube-api-access-sc5wx\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.359288 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.359222    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461dd94-363f-45bb-945f-842cadbf52cd-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.359288 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.359222    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa58e4e-47ba-478f-9223-ae269e303f4b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4aa58e4e-47ba-478f-9223-ae269e303f4b" (UID: "4aa58e4e-47ba-478f-9223-ae269e303f4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:22:07.360913 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.360887    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa58e4e-47ba-478f-9223-ae269e303f4b-kube-api-access-cksfn" (OuterVolumeSpecName: "kube-api-access-cksfn") pod "4aa58e4e-47ba-478f-9223-ae269e303f4b" (UID: "4aa58e4e-47ba-478f-9223-ae269e303f4b"). InnerVolumeSpecName "kube-api-access-cksfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:22:07.460079 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.460043    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa58e4e-47ba-478f-9223-ae269e303f4b-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.460079 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.460069    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cksfn\" (UniqueName: \"kubernetes.io/projected/4aa58e4e-47ba-478f-9223-ae269e303f4b-kube-api-access-cksfn\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:22:07.659631 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.659530    2556 generic.go:356] "Generic (PLEG): container finished" podID="4aa58e4e-47ba-478f-9223-ae269e303f4b" containerID="6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66" exitCode=0
Sep 14 08:22:07.659631 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.659599    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"
Sep 14 08:22:07.659631 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.659617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn" event={"ID":"4aa58e4e-47ba-478f-9223-ae269e303f4b","Type":"ContainerDied","Data":"6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66"}
Sep 14 08:22:07.659907 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.659669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn" event={"ID":"4aa58e4e-47ba-478f-9223-ae269e303f4b","Type":"ContainerDied","Data":"5bb02354b8b735a3e70b57b4e48899d022d6552157383a0e314e327421b0172d"}
Sep 14 08:22:07.659907 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.659700    2556 scope.go:117] "RemoveContainer" containerID="6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66"
Sep 14 08:22:07.661044 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.661016    2556 generic.go:356] "Generic (PLEG): container finished" podID="5461dd94-363f-45bb-945f-842cadbf52cd" containerID="d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8" exitCode=0
Sep 14 08:22:07.661177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.661055    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"
Sep 14 08:22:07.661177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.661096    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47" event={"ID":"5461dd94-363f-45bb-945f-842cadbf52cd","Type":"ContainerDied","Data":"d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8"}
Sep 14 08:22:07.661177 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.661158    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47" event={"ID":"5461dd94-363f-45bb-945f-842cadbf52cd","Type":"ContainerDied","Data":"33e96733719522c02c0aac736832c920b62939498aeaaa9a5d6da47f4aca50c5"}
Sep 14 08:22:07.669527 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.669430    2556 scope.go:117] "RemoveContainer" containerID="6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66"
Sep 14 08:22:07.669797 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:22:07.669772    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66\": container with ID starting with 6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66 not found: ID does not exist" containerID="6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66"
Sep 14 08:22:07.669878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.669807    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66"} err="failed to get container status \"6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66\": rpc error: code = NotFound desc = could not find container \"6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66\": container with ID starting with 6870be3f8c21b1425554de5fa58917356ba1dafd63aa2b0305992a6b94125f66 not found: ID does not exist"
Sep 14 08:22:07.669878 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.669830    2556 scope.go:117] "RemoveContainer" containerID="d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8"
Sep 14 08:22:07.678197 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.678181    2556 scope.go:117] "RemoveContainer" containerID="d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8"
Sep 14 08:22:07.678462 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:22:07.678444    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8\": container with ID starting with d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8 not found: ID does not exist" containerID="d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8"
Sep 14 08:22:07.678539 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.678477    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8"} err="failed to get container status \"d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8\": rpc error: code = NotFound desc = could not find container \"d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8\": container with ID starting with d192a2af5979010736b8ee5dce77aca60df4c61d658e8171e030de00655e28c8 not found: ID does not exist"
Sep 14 08:22:07.685302 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.685281    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"]
Sep 14 08:22:07.694906 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.692232    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-03d6fa-7f699884fc-qtd47"]
Sep 14 08:22:07.703152 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.703129    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"]
Sep 14 08:22:07.706755 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:07.706730    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-03d6fa-6fbfb6df7d-lzxdn"]
Sep 14 08:22:08.020524 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:08.020488    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aa58e4e-47ba-478f-9223-ae269e303f4b" path="/var/lib/kubelet/pods/4aa58e4e-47ba-478f-9223-ae269e303f4b/volumes"
Sep 14 08:22:08.020901 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:22:08.020842    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5461dd94-363f-45bb-945f-842cadbf52cd" path="/var/lib/kubelet/pods/5461dd94-363f-45bb-945f-842cadbf52cd/volumes"
Sep 14 08:23:04.337029 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:04.336992    2556 scope.go:117] "RemoveContainer" containerID="b2f5e2a66ec5e32c91374d3d58b57d4e4115530ab9f20ddd42e04fc87e022828"
Sep 14 08:23:20.205030 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.204995    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"]
Sep 14 08:23:20.205600 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.205255    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh" podUID="869596d6-ab63-4615-8c1f-8f2853363220" containerName="maas-api" containerID="cri-o://9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9" gracePeriod=30
Sep 14 08:23:20.256384 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.256347    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"]
Sep 14 08:23:20.256677 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.256622    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp" podUID="f13268f1-24a6-414a-8d74-96649783b907" containerName="payload-processing" containerID="cri-o://381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd" gracePeriod=30
Sep 14 08:23:20.504161 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.504132    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:23:20.515583 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.515559    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:23:20.605047 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605011    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869596d6-ab63-4615-8c1f-8f2853363220-openshift-service-ca\") pod \"869596d6-ab63-4615-8c1f-8f2853363220\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") "
Sep 14 08:23:20.605047 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605047    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-metrics-certs\") pod \"869596d6-ab63-4615-8c1f-8f2853363220\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") "
Sep 14 08:23:20.605260 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605178    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-maas-api-tls\") pod \"869596d6-ab63-4615-8c1f-8f2853363220\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") "
Sep 14 08:23:20.605260 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605219    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfnhm\" (UniqueName: \"kubernetes.io/projected/869596d6-ab63-4615-8c1f-8f2853363220-kube-api-access-hfnhm\") pod \"869596d6-ab63-4615-8c1f-8f2853363220\" (UID: \"869596d6-ab63-4615-8c1f-8f2853363220\") "
Sep 14 08:23:20.605260 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605249    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q25m\" (UniqueName: \"kubernetes.io/projected/f13268f1-24a6-414a-8d74-96649783b907-kube-api-access-6q25m\") pod \"f13268f1-24a6-414a-8d74-96649783b907\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") "
Sep 14 08:23:20.605422 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605265    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f13268f1-24a6-414a-8d74-96649783b907-plugins-config-volume\") pod \"f13268f1-24a6-414a-8d74-96649783b907\" (UID: \"f13268f1-24a6-414a-8d74-96649783b907\") "
Sep 14 08:23:20.605538 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605483    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/869596d6-ab63-4615-8c1f-8f2853363220-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "869596d6-ab63-4615-8c1f-8f2853363220" (UID: "869596d6-ab63-4615-8c1f-8f2853363220"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:20.605810 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605602    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869596d6-ab63-4615-8c1f-8f2853363220-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.605890 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.605858    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13268f1-24a6-414a-8d74-96649783b907-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f13268f1-24a6-414a-8d74-96649783b907" (UID: "f13268f1-24a6-414a-8d74-96649783b907"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:20.607434 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.607404    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869596d6-ab63-4615-8c1f-8f2853363220-kube-api-access-hfnhm" (OuterVolumeSpecName: "kube-api-access-hfnhm") pod "869596d6-ab63-4615-8c1f-8f2853363220" (UID: "869596d6-ab63-4615-8c1f-8f2853363220"). InnerVolumeSpecName "kube-api-access-hfnhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:20.607565 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.607486    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "869596d6-ab63-4615-8c1f-8f2853363220" (UID: "869596d6-ab63-4615-8c1f-8f2853363220"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:23:20.607787 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.607761    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13268f1-24a6-414a-8d74-96649783b907-kube-api-access-6q25m" (OuterVolumeSpecName: "kube-api-access-6q25m") pod "f13268f1-24a6-414a-8d74-96649783b907" (UID: "f13268f1-24a6-414a-8d74-96649783b907"). InnerVolumeSpecName "kube-api-access-6q25m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:20.607890 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.607817    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "869596d6-ab63-4615-8c1f-8f2853363220" (UID: "869596d6-ab63-4615-8c1f-8f2853363220"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:23:20.706431 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.706394    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.706431 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.706425    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfnhm\" (UniqueName: \"kubernetes.io/projected/869596d6-ab63-4615-8c1f-8f2853363220-kube-api-access-hfnhm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.706431 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.706436    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q25m\" (UniqueName: \"kubernetes.io/projected/f13268f1-24a6-414a-8d74-96649783b907-kube-api-access-6q25m\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.706664 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.706444    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f13268f1-24a6-414a-8d74-96649783b907-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.706664 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:20.706454    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869596d6-ab63-4615-8c1f-8f2853363220-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:21.061809 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.061770    2556 generic.go:356] "Generic (PLEG): container finished" podID="f13268f1-24a6-414a-8d74-96649783b907" containerID="381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd" exitCode=0
Sep 14 08:23:21.061953 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.061821    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"
Sep 14 08:23:21.061953 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.061858    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp" event={"ID":"f13268f1-24a6-414a-8d74-96649783b907","Type":"ContainerDied","Data":"381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd"}
Sep 14 08:23:21.061953 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.061916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp" event={"ID":"f13268f1-24a6-414a-8d74-96649783b907","Type":"ContainerDied","Data":"ce53d38ea052ae5047c4475c6ffa5aad540313e58986c5a389464e78b7c18ea7"}
Sep 14 08:23:21.061953 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.061939    2556 scope.go:117] "RemoveContainer" containerID="381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd"
Sep 14 08:23:21.063515 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.063491    2556 generic.go:356] "Generic (PLEG): container finished" podID="869596d6-ab63-4615-8c1f-8f2853363220" containerID="9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9" exitCode=0
Sep 14 08:23:21.063620 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.063526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh" event={"ID":"869596d6-ab63-4615-8c1f-8f2853363220","Type":"ContainerDied","Data":"9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9"}
Sep 14 08:23:21.063620 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.063543    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"
Sep 14 08:23:21.063620 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.063555    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh" event={"ID":"869596d6-ab63-4615-8c1f-8f2853363220","Type":"ContainerDied","Data":"6ec325aef753d625f0661e1cd32affa48ab4e1deb3cf83f6511ce9569dbf9c27"}
Sep 14 08:23:21.073939 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.073925    2556 scope.go:117] "RemoveContainer" containerID="381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd"
Sep 14 08:23:21.074271 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:23:21.074245    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd\": container with ID starting with 381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd not found: ID does not exist" containerID="381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd"
Sep 14 08:23:21.074421 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.074274    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd"} err="failed to get container status \"381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd\": rpc error: code = NotFound desc = could not find container \"381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd\": container with ID starting with 381a367f22af2f8eec56592d75b6e4bb029840d40bd87fa107a9c6cda5a26dbd not found: ID does not exist"
Sep 14 08:23:21.074421 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.074290    2556 scope.go:117] "RemoveContainer" containerID="9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9"
Sep 14 08:23:21.082566 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.082544    2556 scope.go:117] "RemoveContainer" containerID="9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9"
Sep 14 08:23:21.082836 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:23:21.082818    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9\": container with ID starting with 9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9 not found: ID does not exist" containerID="9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9"
Sep 14 08:23:21.082882 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.082844    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9"} err="failed to get container status \"9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9\": rpc error: code = NotFound desc = could not find container \"9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9\": container with ID starting with 9d67d06d8d57186429fe05149dd3911209559947c4d993c4262d9dad42d469a9 not found: ID does not exist"
Sep 14 08:23:21.084292 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.084273    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"]
Sep 14 08:23:21.089801 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.089781    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7bcf7b-6d8675d48f-526cp"]
Sep 14 08:23:21.098394 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.098371    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"]
Sep 14 08:23:21.104144 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:21.104125    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bcf7b-7596c76f5f-lbpvh"]
Sep 14 08:23:22.019430 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:22.019393    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="869596d6-ab63-4615-8c1f-8f2853363220" path="/var/lib/kubelet/pods/869596d6-ab63-4615-8c1f-8f2853363220/volumes"
Sep 14 08:23:22.019812 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:22.019728    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13268f1-24a6-414a-8d74-96649783b907" path="/var/lib/kubelet/pods/f13268f1-24a6-414a-8d74-96649783b907/volumes"
Sep 14 08:23:37.778789 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:37.778715    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"]
Sep 14 08:23:37.782284 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:37.782252    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-919cc9-29822895-lhmjv"]
Sep 14 08:23:37.799529 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:37.799483    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"]
Sep 14 08:23:37.799786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:37.799733    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh" podUID="e0bd445e-f35f-47de-8e52-e806d9c3e4c8" containerName="payload-processing" containerID="cri-o://2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0" gracePeriod=30
Sep 14 08:23:38.021475 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.021435    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06878ac5-98fb-4b1c-8486-a8486ec786d3" path="/var/lib/kubelet/pods/06878ac5-98fb-4b1c-8486-a8486ec786d3/volumes"
Sep 14 08:23:38.056205 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.056180    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:23:38.148224 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.148192    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-plugins-config-volume\") pod \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") "
Sep 14 08:23:38.148405 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.148231    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tdh4\" (UniqueName: \"kubernetes.io/projected/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-kube-api-access-9tdh4\") pod \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\" (UID: \"e0bd445e-f35f-47de-8e52-e806d9c3e4c8\") "
Sep 14 08:23:38.148611 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.148585    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0bd445e-f35f-47de-8e52-e806d9c3e4c8" (UID: "e0bd445e-f35f-47de-8e52-e806d9c3e4c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:38.150255 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.150233    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-kube-api-access-9tdh4" (OuterVolumeSpecName: "kube-api-access-9tdh4") pod "e0bd445e-f35f-47de-8e52-e806d9c3e4c8" (UID: "e0bd445e-f35f-47de-8e52-e806d9c3e4c8"). InnerVolumeSpecName "kube-api-access-9tdh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:38.158637 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.158613    2556 generic.go:356] "Generic (PLEG): container finished" podID="e0bd445e-f35f-47de-8e52-e806d9c3e4c8" containerID="2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0" exitCode=0
Sep 14 08:23:38.158732 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.158675    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"
Sep 14 08:23:38.158732 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.158696    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh" event={"ID":"e0bd445e-f35f-47de-8e52-e806d9c3e4c8","Type":"ContainerDied","Data":"2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0"}
Sep 14 08:23:38.158822 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.158752    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh" event={"ID":"e0bd445e-f35f-47de-8e52-e806d9c3e4c8","Type":"ContainerDied","Data":"a8af011d72ca9b84dcc176290e4195840a2afbb10c8e88c292d0834efefff5d1"}
Sep 14 08:23:38.158822 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.158779    2556 scope.go:117] "RemoveContainer" containerID="2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0"
Sep 14 08:23:38.173600 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.173579    2556 scope.go:117] "RemoveContainer" containerID="2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0"
Sep 14 08:23:38.173924 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:23:38.173902    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0\": container with ID starting with 2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0 not found: ID does not exist" containerID="2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0"
Sep 14 08:23:38.174031 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.173932    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0"} err="failed to get container status \"2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0\": rpc error: code = NotFound desc = could not find container \"2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0\": container with ID starting with 2bab344117d011c87ee981a094684b88b77c37713fbe908d08923c272ddf26d0 not found: ID does not exist"
Sep 14 08:23:38.184587 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.184561    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"]
Sep 14 08:23:38.186712 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.186693    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-919cc9-76cc788cdc-zh8zh"]
Sep 14 08:23:38.249256 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.249223    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.249256 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:38.249251    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tdh4\" (UniqueName: \"kubernetes.io/projected/e0bd445e-f35f-47de-8e52-e806d9c3e4c8-kube-api-access-9tdh4\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:23:40.018785 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:40.018745    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0bd445e-f35f-47de-8e52-e806d9c3e4c8" path="/var/lib/kubelet/pods/e0bd445e-f35f-47de-8e52-e806d9c3e4c8/volumes"
Sep 14 08:23:55.424733 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:55.424696    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"]
Sep 14 08:23:55.427414 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:55.427383    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fc0d6e-29822895-mzt4c"]
Sep 14 08:23:56.018844 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:23:56.018804    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25282b06-1c47-4813-ade6-5c75003d69fa" path="/var/lib/kubelet/pods/25282b06-1c47-4813-ade6-5c75003d69fa/volumes"
Sep 14 08:24:04.364599 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:04.364563    2556 scope.go:117] "RemoveContainer" containerID="4a69f91567772de2efaa6d306db0837ee01b7cee0aa81bfe6506dd96b5386022"
Sep 14 08:24:04.373459 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:04.373431    2556 scope.go:117] "RemoveContainer" containerID="3345e4fa493f077402ef2bb0f31c81ddaf78ad30dc62cee828a2df2b501fbe96"
Sep 14 08:24:05.028345 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.028311    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8944c479-b5jvw"]
Sep 14 08:24:05.028593 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.028545    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c8944c479-b5jvw" podUID="3627632d-b644-4597-8148-cd18640973ff" containerName="manager" containerID="cri-o://1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390" gracePeriod=30
Sep 14 08:24:05.279718 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.279660    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 08:24:05.307789 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.307756    2556 generic.go:356] "Generic (PLEG): container finished" podID="3627632d-b644-4597-8148-cd18640973ff" containerID="1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390" exitCode=0
Sep 14 08:24:05.307957 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.307794    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-b5jvw"
Sep 14 08:24:05.307957 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.307834    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-b5jvw" event={"ID":"3627632d-b644-4597-8148-cd18640973ff","Type":"ContainerDied","Data":"1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390"}
Sep 14 08:24:05.307957 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.307890    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-b5jvw" event={"ID":"3627632d-b644-4597-8148-cd18640973ff","Type":"ContainerDied","Data":"2d93c8387b31e82ffb8e2c0f1d723c6db2dfa9f5a1e9ec29c03ea435df71a4b2"}
Sep 14 08:24:05.307957 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.307918    2556 scope.go:117] "RemoveContainer" containerID="1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390"
Sep 14 08:24:05.317711 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.317679    2556 scope.go:117] "RemoveContainer" containerID="1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390"
Sep 14 08:24:05.318067 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:24:05.318040    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390\": container with ID starting with 1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390 not found: ID does not exist" containerID="1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390"
Sep 14 08:24:05.318147 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.318079    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390"} err="failed to get container status \"1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390\": rpc error: code = NotFound desc = could not find container \"1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390\": container with ID starting with 1509b2700a696406794c3bcf432696ca2baa0853132a976d4a8d2aa689d35390 not found: ID does not exist"
Sep 14 08:24:05.374950 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.374917    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-metrics-certs\") pod \"3627632d-b644-4597-8148-cd18640973ff\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") "
Sep 14 08:24:05.375348 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.375021    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26rkn\" (UniqueName: \"kubernetes.io/projected/3627632d-b644-4597-8148-cd18640973ff-kube-api-access-26rkn\") pod \"3627632d-b644-4597-8148-cd18640973ff\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") "
Sep 14 08:24:05.375348 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.375058    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert\") pod \"3627632d-b644-4597-8148-cd18640973ff\" (UID: \"3627632d-b644-4597-8148-cd18640973ff\") "
Sep 14 08:24:05.376978 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.376929    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3627632d-b644-4597-8148-cd18640973ff" (UID: "3627632d-b644-4597-8148-cd18640973ff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:24:05.377077 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.376980    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3627632d-b644-4597-8148-cd18640973ff-kube-api-access-26rkn" (OuterVolumeSpecName: "kube-api-access-26rkn") pod "3627632d-b644-4597-8148-cd18640973ff" (UID: "3627632d-b644-4597-8148-cd18640973ff"). InnerVolumeSpecName "kube-api-access-26rkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:24:05.377077 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.376979    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3627632d-b644-4597-8148-cd18640973ff" (UID: "3627632d-b644-4597-8148-cd18640973ff"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:24:05.475936 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.475898    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-webhook-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:24:05.475936 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.475928    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3627632d-b644-4597-8148-cd18640973ff-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:24:05.475936 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.475942    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26rkn\" (UniqueName: \"kubernetes.io/projected/3627632d-b644-4597-8148-cd18640973ff-kube-api-access-26rkn\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:24:05.628396 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.628364    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8944c479-b5jvw"]
Sep 14 08:24:05.632061 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:05.632033    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c8944c479-b5jvw"]
Sep 14 08:24:06.019270 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:24:06.019238    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3627632d-b644-4597-8148-cd18640973ff" path="/var/lib/kubelet/pods/3627632d-b644-4597-8148-cd18640973ff/volumes"
Sep 14 08:25:06.483165 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483124    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8944c479-l2tsx"]
Sep 14 08:25:06.483597 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483578    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4698bb-1a45-456e-bfbc-6a504648fc2d" containerName="maas-api"
Sep 14 08:25:06.483642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483599    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4698bb-1a45-456e-bfbc-6a504648fc2d" containerName="maas-api"
Sep 14 08:25:06.483642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483608    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3627632d-b644-4597-8148-cd18640973ff" containerName="manager"
Sep 14 08:25:06.483642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483614    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3627632d-b644-4597-8148-cd18640973ff" containerName="manager"
Sep 14 08:25:06.483642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483622    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13268f1-24a6-414a-8d74-96649783b907" containerName="payload-processing"
Sep 14 08:25:06.483642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483629    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13268f1-24a6-414a-8d74-96649783b907" containerName="payload-processing"
Sep 14 08:25:06.483642 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483635    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aa58e4e-47ba-478f-9223-ae269e303f4b" containerName="payload-pre-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483648    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aa58e4e-47ba-478f-9223-ae269e303f4b" containerName="payload-pre-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483656    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="869596d6-ab63-4615-8c1f-8f2853363220" containerName="maas-api"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483664    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="869596d6-ab63-4615-8c1f-8f2853363220" containerName="maas-api"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483679    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0bd445e-f35f-47de-8e52-e806d9c3e4c8" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483685    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0bd445e-f35f-47de-8e52-e806d9c3e4c8" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483694    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5461dd94-363f-45bb-945f-842cadbf52cd" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483700    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5461dd94-363f-45bb-945f-842cadbf52cd" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483754    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4698bb-1a45-456e-bfbc-6a504648fc2d" containerName="maas-api"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483762    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0bd445e-f35f-47de-8e52-e806d9c3e4c8" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483768    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3627632d-b644-4597-8148-cd18640973ff" containerName="manager"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483775    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13268f1-24a6-414a-8d74-96649783b907" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483780    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5461dd94-363f-45bb-945f-842cadbf52cd" containerName="payload-processing"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483785    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="869596d6-ab63-4615-8c1f-8f2853363220" containerName="maas-api"
Sep 14 08:25:06.483826 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.483790    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aa58e4e-47ba-478f-9223-ae269e303f4b" containerName="payload-pre-processing"
Sep 14 08:25:06.487165 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.487147    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.489979 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.489947    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 08:25:06.490098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.489993    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 08:25:06.490098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.489951    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lrt2\""
Sep 14 08:25:06.490217 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.489947    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 08:25:06.497332 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.497310    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8944c479-l2tsx"]
Sep 14 08:25:06.586985 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.586932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-webhook-cert\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.587165 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.587055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2bt5\" (UniqueName: \"kubernetes.io/projected/743d366b-9a79-4c49-a01c-6865ccbc71b8-kube-api-access-q2bt5\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.587165 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.587115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-metrics-certs\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.687943 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.687910    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2bt5\" (UniqueName: \"kubernetes.io/projected/743d366b-9a79-4c49-a01c-6865ccbc71b8-kube-api-access-q2bt5\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.688148 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.687986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-metrics-certs\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.688148 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.688052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-webhook-cert\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.690421 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.690397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-metrics-certs\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.690565 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.690542    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-webhook-cert\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.697006 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.696957    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2bt5\" (UniqueName: \"kubernetes.io/projected/743d366b-9a79-4c49-a01c-6865ccbc71b8-kube-api-access-q2bt5\") pod \"maas-controller-6c8944c479-l2tsx\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") " pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:06.798243 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:06.798151    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:07.069067 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:07.069036    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8944c479-l2tsx"]
Sep 14 08:25:07.071048 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:25:07.071021    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743d366b_9a79_4c49_a01c_6865ccbc71b8.slice/crio-6bd09e2b2b90e21aef62a044aff272eef31a278fdd01d176c2651f4b63de5cd8 WatchSource:0}: Error finding container 6bd09e2b2b90e21aef62a044aff272eef31a278fdd01d176c2651f4b63de5cd8: Status 404 returned error can't find the container with id 6bd09e2b2b90e21aef62a044aff272eef31a278fdd01d176c2651f4b63de5cd8
Sep 14 08:25:07.072708 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:07.072690    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:25:07.634236 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:07.634204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-l2tsx" event={"ID":"743d366b-9a79-4c49-a01c-6865ccbc71b8","Type":"ContainerStarted","Data":"6bd09e2b2b90e21aef62a044aff272eef31a278fdd01d176c2651f4b63de5cd8"}
Sep 14 08:25:08.641841 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:08.641801    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-l2tsx" event={"ID":"743d366b-9a79-4c49-a01c-6865ccbc71b8","Type":"ContainerStarted","Data":"76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e"}
Sep 14 08:25:08.642345 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:08.641928    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:08.659929 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:08.659881    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8944c479-l2tsx" podStartSLOduration=1.871898437 podStartE2EDuration="2.659867266s" totalImagesPullingTime="787.968829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:25:08.657398737 +0000 UTC m=+2165.156791528" watchObservedRunningTime="2026-09-14 08:25:08.659867266 +0000 UTC m=+2165.159260086"
Sep 14 08:25:19.649690 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:19.649660    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:26.939691 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:26.939643    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8944c479-l2tsx"]
Sep 14 08:25:26.940308 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:26.939851    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c8944c479-l2tsx" podUID="743d366b-9a79-4c49-a01c-6865ccbc71b8" containerName="manager" containerID="cri-o://76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e" gracePeriod=30
Sep 14 08:25:27.181762 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.181737    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:27.273667 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.273582    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2bt5\" (UniqueName: \"kubernetes.io/projected/743d366b-9a79-4c49-a01c-6865ccbc71b8-kube-api-access-q2bt5\") pod \"743d366b-9a79-4c49-a01c-6865ccbc71b8\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") "
Sep 14 08:25:27.273667 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.273641    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-webhook-cert\") pod \"743d366b-9a79-4c49-a01c-6865ccbc71b8\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") "
Sep 14 08:25:27.273916 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.273677    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-metrics-certs\") pod \"743d366b-9a79-4c49-a01c-6865ccbc71b8\" (UID: \"743d366b-9a79-4c49-a01c-6865ccbc71b8\") "
Sep 14 08:25:27.275746 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.275712    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "743d366b-9a79-4c49-a01c-6865ccbc71b8" (UID: "743d366b-9a79-4c49-a01c-6865ccbc71b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:25:27.275862 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.275768    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/743d366b-9a79-4c49-a01c-6865ccbc71b8-kube-api-access-q2bt5" (OuterVolumeSpecName: "kube-api-access-q2bt5") pod "743d366b-9a79-4c49-a01c-6865ccbc71b8" (UID: "743d366b-9a79-4c49-a01c-6865ccbc71b8"). InnerVolumeSpecName "kube-api-access-q2bt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:25:27.275862 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.275775    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "743d366b-9a79-4c49-a01c-6865ccbc71b8" (UID: "743d366b-9a79-4c49-a01c-6865ccbc71b8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:25:27.374854 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.374817    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:25:27.374854 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.374848    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2bt5\" (UniqueName: \"kubernetes.io/projected/743d366b-9a79-4c49-a01c-6865ccbc71b8-kube-api-access-q2bt5\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:25:27.374854 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.374861    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/743d366b-9a79-4c49-a01c-6865ccbc71b8-webhook-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:25:27.740485 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.740448    2556 generic.go:356] "Generic (PLEG): container finished" podID="743d366b-9a79-4c49-a01c-6865ccbc71b8" containerID="76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e" exitCode=0
Sep 14 08:25:27.740674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.740489    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-l2tsx"
Sep 14 08:25:27.740674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.740536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-l2tsx" event={"ID":"743d366b-9a79-4c49-a01c-6865ccbc71b8","Type":"ContainerDied","Data":"76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e"}
Sep 14 08:25:27.740674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.740587    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-l2tsx" event={"ID":"743d366b-9a79-4c49-a01c-6865ccbc71b8","Type":"ContainerDied","Data":"6bd09e2b2b90e21aef62a044aff272eef31a278fdd01d176c2651f4b63de5cd8"}
Sep 14 08:25:27.740674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.740610    2556 scope.go:117] "RemoveContainer" containerID="76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e"
Sep 14 08:25:27.750098 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.750077    2556 scope.go:117] "RemoveContainer" containerID="76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e"
Sep 14 08:25:27.750347 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:25:27.750332    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e\": container with ID starting with 76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e not found: ID does not exist" containerID="76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e"
Sep 14 08:25:27.750399 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.750354    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e"} err="failed to get container status \"76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e\": rpc error: code = NotFound desc = could not find container \"76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e\": container with ID starting with 76590cc65dce2b3b63e224ba66780d0a201887229286e8cf613955b8f0e6cf9e not found: ID does not exist"
Sep 14 08:25:27.761872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.761849    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8944c479-l2tsx"]
Sep 14 08:25:27.763748 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:27.763727    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c8944c479-l2tsx"]
Sep 14 08:25:28.018203 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.018125    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="743d366b-9a79-4c49-a01c-6865ccbc71b8" path="/var/lib/kubelet/pods/743d366b-9a79-4c49-a01c-6865ccbc71b8/volumes"
Sep 14 08:25:28.872888 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.872854    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8944c479-fflxt"]
Sep 14 08:25:28.873212 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.873199    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="743d366b-9a79-4c49-a01c-6865ccbc71b8" containerName="manager"
Sep 14 08:25:28.873259 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.873214    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="743d366b-9a79-4c49-a01c-6865ccbc71b8" containerName="manager"
Sep 14 08:25:28.873295 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.873283    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="743d366b-9a79-4c49-a01c-6865ccbc71b8" containerName="manager"
Sep 14 08:25:28.877654 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.877634    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:28.880315 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.880289    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lrt2\""
Sep 14 08:25:28.880315 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.880302    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 08:25:28.880490 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.880290    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 08:25:28.880490 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.880328    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 08:25:28.884559 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.884538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8944c479-fflxt"]
Sep 14 08:25:28.987699 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.987661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac0e195d-f177-42ba-97f8-9e88833a15ff-webhook-cert\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:28.987901 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.987707    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qtk4\" (UniqueName: \"kubernetes.io/projected/ac0e195d-f177-42ba-97f8-9e88833a15ff-kube-api-access-4qtk4\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:28.987901 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:28.987833    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac0e195d-f177-42ba-97f8-9e88833a15ff-metrics-certs\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.088260 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.088219    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac0e195d-f177-42ba-97f8-9e88833a15ff-metrics-certs\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.088752 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.088270    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac0e195d-f177-42ba-97f8-9e88833a15ff-webhook-cert\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.088752 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.088295    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qtk4\" (UniqueName: \"kubernetes.io/projected/ac0e195d-f177-42ba-97f8-9e88833a15ff-kube-api-access-4qtk4\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.090867 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.090834    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac0e195d-f177-42ba-97f8-9e88833a15ff-webhook-cert\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.090988 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.090890    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac0e195d-f177-42ba-97f8-9e88833a15ff-metrics-certs\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.095478 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.095449    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qtk4\" (UniqueName: \"kubernetes.io/projected/ac0e195d-f177-42ba-97f8-9e88833a15ff-kube-api-access-4qtk4\") pod \"maas-controller-6c8944c479-fflxt\" (UID: \"ac0e195d-f177-42ba-97f8-9e88833a15ff\") " pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.189165 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.189127    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:29.460134 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.460103    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8944c479-fflxt"]
Sep 14 08:25:29.461521 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:25:29.461485    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0e195d_f177_42ba_97f8_9e88833a15ff.slice/crio-c271f77c41a8405e23ab45f3c606403c80b3ddc12c937c4099f8915f384d72f0 WatchSource:0}: Error finding container c271f77c41a8405e23ab45f3c606403c80b3ddc12c937c4099f8915f384d72f0: Status 404 returned error can't find the container with id c271f77c41a8405e23ab45f3c606403c80b3ddc12c937c4099f8915f384d72f0
Sep 14 08:25:29.752047 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:29.751945    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-fflxt" event={"ID":"ac0e195d-f177-42ba-97f8-9e88833a15ff","Type":"ContainerStarted","Data":"c271f77c41a8405e23ab45f3c606403c80b3ddc12c937c4099f8915f384d72f0"}
Sep 14 08:25:30.761399 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:30.761359    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8944c479-fflxt" event={"ID":"ac0e195d-f177-42ba-97f8-9e88833a15ff","Type":"ContainerStarted","Data":"1d99d45fda48a827405b9bb79b722fc6ce5e5d88bf3b57d23f057201f7812e1c"}
Sep 14 08:25:30.761868 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:30.761453    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:25:30.778049 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:30.777997    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8944c479-fflxt" podStartSLOduration=2.425273125 podStartE2EDuration="2.777956048s" totalImagesPullingTime="352.682923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:25:30.77636471 +0000 UTC m=+2187.275757519" watchObservedRunningTime="2026-09-14 08:25:30.777956048 +0000 UTC m=+2187.277348826"
Sep 14 08:25:41.769559 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:25:41.769524    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8944c479-fflxt"
Sep 14 08:29:42.216738 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:42.216642    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"]
Sep 14 08:29:42.217280 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:42.216918    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj" podUID="55ab9891-a870-4b86-a3cd-2bfb7a5b674f" containerName="maas-api" containerID="cri-o://4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b" gracePeriod=30
Sep 14 08:29:43.467436 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.467409    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 08:29:43.520825 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.520780    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-kube-api-access-t7qsg\") pod \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") "
Sep 14 08:29:43.521030 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.520850    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-metrics-certs\") pod \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") "
Sep 14 08:29:43.521030 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.520958    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-maas-api-tls\") pod \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") "
Sep 14 08:29:43.521030 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.521019    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-openshift-service-ca\") pod \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\" (UID: \"55ab9891-a870-4b86-a3cd-2bfb7a5b674f\") "
Sep 14 08:29:43.521372 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.521337    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55ab9891-a870-4b86-a3cd-2bfb7a5b674f" (UID: "55ab9891-a870-4b86-a3cd-2bfb7a5b674f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:29:43.522941 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.522917    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55ab9891-a870-4b86-a3cd-2bfb7a5b674f" (UID: "55ab9891-a870-4b86-a3cd-2bfb7a5b674f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:29:43.523073 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.522942    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-kube-api-access-t7qsg" (OuterVolumeSpecName: "kube-api-access-t7qsg") pod "55ab9891-a870-4b86-a3cd-2bfb7a5b674f" (UID: "55ab9891-a870-4b86-a3cd-2bfb7a5b674f"). InnerVolumeSpecName "kube-api-access-t7qsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:29:43.523073 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.523034    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55ab9891-a870-4b86-a3cd-2bfb7a5b674f" (UID: "55ab9891-a870-4b86-a3cd-2bfb7a5b674f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:29:43.621837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.621745    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-maas-api-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:29:43.621837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.621778    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-openshift-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:29:43.621837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.621789    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-kube-api-access-t7qsg\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:29:43.621837 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:43.621798    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ab9891-a870-4b86-a3cd-2bfb7a5b674f-metrics-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Sep 14 08:29:44.052416 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.052386    2556 generic.go:356] "Generic (PLEG): container finished" podID="55ab9891-a870-4b86-a3cd-2bfb7a5b674f" containerID="4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b" exitCode=0
Sep 14 08:29:44.052602 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.052434    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"
Sep 14 08:29:44.052602 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.052478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj" event={"ID":"55ab9891-a870-4b86-a3cd-2bfb7a5b674f","Type":"ContainerDied","Data":"4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b"}
Sep 14 08:29:44.052602 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.052535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj" event={"ID":"55ab9891-a870-4b86-a3cd-2bfb7a5b674f","Type":"ContainerDied","Data":"20374b6a8f0ff6837847cbab8bc543b35b272ad7b15bef97530672aa16bb76a8"}
Sep 14 08:29:44.052602 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.052563    2556 scope.go:117] "RemoveContainer" containerID="4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b"
Sep 14 08:29:44.061698 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.061678    2556 scope.go:117] "RemoveContainer" containerID="4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b"
Sep 14 08:29:44.061985 ip-10-0-131-236 kubenswrapper[2556]: E0914 08:29:44.061944    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b\": container with ID starting with 4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b not found: ID does not exist" containerID="4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b"
Sep 14 08:29:44.062063 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.061998    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b"} err="failed to get container status \"4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b\": rpc error: code = NotFound desc = could not find container \"4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b\": container with ID starting with 4f5c8a451f815da1ce95a11bfa54a4865a70d92d0fee874c239707beba8cf33b not found: ID does not exist"
Sep 14 08:29:44.067460 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.067436    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"]
Sep 14 08:29:44.069433 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:44.069411    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f5c8b48b7-jcvhj"]
Sep 14 08:29:46.018150 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:29:46.018117    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ab9891-a870-4b86-a3cd-2bfb7a5b674f" path="/var/lib/kubelet/pods/55ab9891-a870-4b86-a3cd-2bfb7a5b674f/volumes"
Sep 14 08:30:00.135562 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.135526    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"]
Sep 14 08:30:00.136069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.135871    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ab9891-a870-4b86-a3cd-2bfb7a5b674f" containerName="maas-api"
Sep 14 08:30:00.136069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.135885    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ab9891-a870-4b86-a3cd-2bfb7a5b674f" containerName="maas-api"
Sep 14 08:30:00.136069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.135992    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ab9891-a870-4b86-a3cd-2bfb7a5b674f" containerName="maas-api"
Sep 14 08:30:00.140450 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.140429    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"
Sep 14 08:30:00.142430 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.142406    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:30:00.147069 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.147045    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"]
Sep 14 08:30:00.152281 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.152254    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"
Sep 14 08:30:00.432695 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.432639    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"]
Sep 14 08:30:00.438263 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:30:00.438234    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefeda655_8e66_4eb1_80dd_2dfd2d6877da.slice/crio-79d4a76deb4edb4b24789a9d90d3f5f99481354bf8b73f3cddfc96327abb3eed WatchSource:0}: Error finding container 79d4a76deb4edb4b24789a9d90d3f5f99481354bf8b73f3cddfc96327abb3eed: Status 404 returned error can't find the container with id 79d4a76deb4edb4b24789a9d90d3f5f99481354bf8b73f3cddfc96327abb3eed
Sep 14 08:30:00.474609 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.474586    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-td4m4_ed06b2ab-ee0b-43d2-8308-b01b4eb31dce/manager/0.log"
Sep 14 08:30:00.728274 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.728194    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c8944c479-fflxt_ac0e195d-f177-42ba-97f8-9e88833a15ff/manager/0.log"
Sep 14 08:30:00.853956 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.853925    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wkflq_054dc377-2db5-4561-ada1-42c434bc070e/server/0.log"
Sep 14 08:30:00.973329 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:00.973294    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g6z28_9662987c-4423-4a86-b32e-5e6b726d2d3a/manager/0.log"
Sep 14 08:30:01.142167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:01.142067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746" event={"ID":"efeda655-8e66-4eb1-80dd-2dfd2d6877da","Type":"ContainerStarted","Data":"fd1c57289c6585773182ced388164f7507bff4dd71844fad89bfd2467ad2b004"}
Sep 14 08:30:01.142167 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:01.142120    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746" event={"ID":"efeda655-8e66-4eb1-80dd-2dfd2d6877da","Type":"ContainerStarted","Data":"79d4a76deb4edb4b24789a9d90d3f5f99481354bf8b73f3cddfc96327abb3eed"}
Sep 14 08:30:01.157833 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:01.157788    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746" podStartSLOduration=1.157771093 podStartE2EDuration="1.157771093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:30:01.157549315 +0000 UTC m=+2457.656942116" watchObservedRunningTime="2026-09-14 08:30:01.157771093 +0000 UTC m=+2457.657163871"
Sep 14 08:30:01.360683 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:01.360660    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-ttn8h_308860a3-0a59-4ba6-8c96-d52b471e2810/copy-manifests/0.log"
Sep 14 08:30:01.367377 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:01.367360    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-ttn8h_308860a3-0a59-4ba6-8c96-d52b471e2810/manager/0.log"
Sep 14 08:30:02.150298 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:02.150261    2556 generic.go:356] "Generic (PLEG): container finished" podID="efeda655-8e66-4eb1-80dd-2dfd2d6877da" containerID="fd1c57289c6585773182ced388164f7507bff4dd71844fad89bfd2467ad2b004" exitCode=0
Sep 14 08:30:02.150687 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:02.150307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746" event={"ID":"efeda655-8e66-4eb1-80dd-2dfd2d6877da","Type":"ContainerDied","Data":"fd1c57289c6585773182ced388164f7507bff4dd71844fad89bfd2467ad2b004"}
Sep 14 08:30:02.172350 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:02.172324    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822910-ct746_efeda655-8e66-4eb1-80dd-2dfd2d6877da/cleanup/0.log"
Sep 14 08:30:02.291870 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:02.291834    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8fsj7_c8782a57-8e69-41b6-b0b4-fc1ebf949cbf/postgres/0.log"
Sep 14 08:30:03.278670 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:03.278643    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"
Sep 14 08:30:03.456632 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:03.456601    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-jtxtz_c7c37f8f-c4bf-46f2-8404-d5421d8594d7/kuadrant-console-plugin/0.log"
Sep 14 08:30:03.804952 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:03.804873    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-xkkbf_e6e14278-6f9d-43f2-b812-59d052d46258/manager/0.log"
Sep 14 08:30:04.162500 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:04.162427    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746"
Sep 14 08:30:04.162500 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:04.162464    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822910-ct746" event={"ID":"efeda655-8e66-4eb1-80dd-2dfd2d6877da","Type":"ContainerDied","Data":"79d4a76deb4edb4b24789a9d90d3f5f99481354bf8b73f3cddfc96327abb3eed"}
Sep 14 08:30:04.162674 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:04.162508    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79d4a76deb4edb4b24789a9d90d3f5f99481354bf8b73f3cddfc96327abb3eed"
Sep 14 08:30:04.779733 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:04.779656    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gnjdc_e86a2a63-e727-45a1-bb6b-4e11e821e409/payload-pre-processing/0.log"
Sep 14 08:30:04.919456 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:04.919423    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q6hxd_1231c9fa-d159-43fd-8c3a-024a981babde/payload-processing/0.log"
Sep 14 08:30:05.759214 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:05.759185    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-286hc_a170840a-585e-42a6-9789-d8478b72f676/main/0.log"
Sep 14 08:30:05.880845 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:05.880813    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw4cc4_613e05d0-bc03-4f82-8375-f4c77f1b72e9/main/0.log"
Sep 14 08:30:12.271515 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.271327    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bqggw/must-gather-8qx5q"]
Sep 14 08:30:12.272004 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.271859    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efeda655-8e66-4eb1-80dd-2dfd2d6877da" containerName="cleanup"
Sep 14 08:30:12.272004 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.271879    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="efeda655-8e66-4eb1-80dd-2dfd2d6877da" containerName="cleanup"
Sep 14 08:30:12.272004 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.271976    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="efeda655-8e66-4eb1-80dd-2dfd2d6877da" containerName="cleanup"
Sep 14 08:30:12.275129 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.275101    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/must-gather-8qx5q"]
Sep 14 08:30:12.275307 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.275186    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.277540 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.277520    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bqggw\"/\"kube-root-ca.crt\""
Sep 14 08:30:12.278316 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.278251    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bqggw\"/\"default-dockercfg-frvbt\""
Sep 14 08:30:12.278316 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.278280    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bqggw\"/\"openshift-service-ca.crt\""
Sep 14 08:30:12.379067 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.379030    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b31204b3-685f-4a65-b1c1-a4c9b2a010cb-must-gather-output\") pod \"must-gather-8qx5q\" (UID: \"b31204b3-685f-4a65-b1c1-a4c9b2a010cb\") " pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.379227 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.379097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c56m9\" (UniqueName: \"kubernetes.io/projected/b31204b3-685f-4a65-b1c1-a4c9b2a010cb-kube-api-access-c56m9\") pod \"must-gather-8qx5q\" (UID: \"b31204b3-685f-4a65-b1c1-a4c9b2a010cb\") " pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.480208 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.480171    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c56m9\" (UniqueName: \"kubernetes.io/projected/b31204b3-685f-4a65-b1c1-a4c9b2a010cb-kube-api-access-c56m9\") pod \"must-gather-8qx5q\" (UID: \"b31204b3-685f-4a65-b1c1-a4c9b2a010cb\") " pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.480374 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.480251    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b31204b3-685f-4a65-b1c1-a4c9b2a010cb-must-gather-output\") pod \"must-gather-8qx5q\" (UID: \"b31204b3-685f-4a65-b1c1-a4c9b2a010cb\") " pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.480590 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.480569    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b31204b3-685f-4a65-b1c1-a4c9b2a010cb-must-gather-output\") pod \"must-gather-8qx5q\" (UID: \"b31204b3-685f-4a65-b1c1-a4c9b2a010cb\") " pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.487886 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.487857    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c56m9\" (UniqueName: \"kubernetes.io/projected/b31204b3-685f-4a65-b1c1-a4c9b2a010cb-kube-api-access-c56m9\") pod \"must-gather-8qx5q\" (UID: \"b31204b3-685f-4a65-b1c1-a4c9b2a010cb\") " pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.586384 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.586297    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/must-gather-8qx5q"
Sep 14 08:30:12.857403 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.857372    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/must-gather-8qx5q"]
Sep 14 08:30:12.859349 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:30:12.859322    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb31204b3_685f_4a65_b1c1_a4c9b2a010cb.slice/crio-2f4b8581502be475a29b48b06c97a2cf5ae5043ad5f471f07d960b10896197d0 WatchSource:0}: Error finding container 2f4b8581502be475a29b48b06c97a2cf5ae5043ad5f471f07d960b10896197d0: Status 404 returned error can't find the container with id 2f4b8581502be475a29b48b06c97a2cf5ae5043ad5f471f07d960b10896197d0
Sep 14 08:30:12.861184 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:12.861165    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:30:13.209358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:13.209325    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/must-gather-8qx5q" event={"ID":"b31204b3-685f-4a65-b1c1-a4c9b2a010cb","Type":"ContainerStarted","Data":"2f4b8581502be475a29b48b06c97a2cf5ae5043ad5f471f07d960b10896197d0"}
Sep 14 08:30:14.217051 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:14.217016    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/must-gather-8qx5q" event={"ID":"b31204b3-685f-4a65-b1c1-a4c9b2a010cb","Type":"ContainerStarted","Data":"12bc1415d07cef93abeeaf08706d2c836c04b81f71d8d95b91822f7a9761af56"}
Sep 14 08:30:14.217507 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:14.217060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/must-gather-8qx5q" event={"ID":"b31204b3-685f-4a65-b1c1-a4c9b2a010cb","Type":"ContainerStarted","Data":"24a7efd9cabae47094e21fe019b2450a24c66adf7cd23bbf8efdf11ec9e8a86d"}
Sep 14 08:30:14.233348 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:14.233285    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bqggw/must-gather-8qx5q" podStartSLOduration=1.32492647 podStartE2EDuration="2.23326161s" totalImagesPullingTime="908.33514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:30:14.230682581 +0000 UTC m=+2470.730075361" watchObservedRunningTime="2026-09-14 08:30:14.23326161 +0000 UTC m=+2470.732654391"
Sep 14 08:30:15.316484 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:15.316446    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tc6ts_60928bfa-f870-48b6-9806-1975c0990803/global-pull-secret-syncer/0.log"
Sep 14 08:30:15.396444 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:15.396410    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wrck8_05bfe619-feda-4437-a91f-4a26bcf42615/konnectivity-agent/0.log"
Sep 14 08:30:15.448623 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:15.448591    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-236.ec2.internal_6b56fcbc9aa34740ef36ad85c9970517/haproxy/0.log"
Sep 14 08:30:19.632174 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:19.632144    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-jtxtz_c7c37f8f-c4bf-46f2-8404-d5421d8594d7/kuadrant-console-plugin/0.log"
Sep 14 08:30:19.784627 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:19.784591    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-xkkbf_e6e14278-6f9d-43f2-b812-59d052d46258/manager/0.log"
Sep 14 08:30:21.561443 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:21.561407    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8f74c88f-57chb_4546a0bc-bfd5-47f7-a0fb-afa108d9f027/metrics-server/0.log"
Sep 14 08:30:21.621769 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:21.621740    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7gjkd_a60e1905-d061-44ce-97f6-a69deaf7bfad/node-exporter/0.log"
Sep 14 08:30:21.644621 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:21.644563    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7gjkd_a60e1905-d061-44ce-97f6-a69deaf7bfad/kube-rbac-proxy/0.log"
Sep 14 08:30:21.668991 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:21.668880    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7gjkd_a60e1905-d061-44ce-97f6-a69deaf7bfad/init-textfile/0.log"
Sep 14 08:30:25.765612 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.765570    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-nstdp_e68e6b4d-e64a-4723-b265-30880b058fdf/volume-data-source-validator/0.log"
Sep 14 08:30:25.780516 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.780480    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"]
Sep 14 08:30:25.784555 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.784526    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.792654 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.792626    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"]
Sep 14 08:30:25.818170 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.818123    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jt8v\" (UniqueName: \"kubernetes.io/projected/07ef37a7-d38f-4e64-a956-6514c9b4426d-kube-api-access-4jt8v\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.818358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.818192    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-podres\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.818358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.818218    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-proc\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.818358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.818277    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-lib-modules\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.818358 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.818327    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-sys\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.919439 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.919395    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jt8v\" (UniqueName: \"kubernetes.io/projected/07ef37a7-d38f-4e64-a956-6514c9b4426d-kube-api-access-4jt8v\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.919733 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.919713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-podres\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.920002 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.919982    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-proc\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.920263 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.920247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-lib-modules\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.920533 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.920516    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-sys\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.920757 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.920109    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-proc\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.920861 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.920444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-lib-modules\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.920942 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.920164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-podres\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.921056 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.920675    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07ef37a7-d38f-4e64-a956-6514c9b4426d-sys\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:25.928591 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:25.928559    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jt8v\" (UniqueName: \"kubernetes.io/projected/07ef37a7-d38f-4e64-a956-6514c9b4426d-kube-api-access-4jt8v\") pod \"perf-node-gather-daemonset-qlfwn\" (UID: \"07ef37a7-d38f-4e64-a956-6514c9b4426d\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:26.100184 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:26.100097    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:26.398872 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:26.398835    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"]
Sep 14 08:30:26.400408 ip-10-0-131-236 kubenswrapper[2556]: W0914 08:30:26.400379    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod07ef37a7_d38f_4e64_a956_6514c9b4426d.slice/crio-9daaf4481c976a8e155526191df48b987d579a758dd3fbf91a07170ee2a038e5 WatchSource:0}: Error finding container 9daaf4481c976a8e155526191df48b987d579a758dd3fbf91a07170ee2a038e5: Status 404 returned error can't find the container with id 9daaf4481c976a8e155526191df48b987d579a758dd3fbf91a07170ee2a038e5
Sep 14 08:30:27.009754 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.009704    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9tqc4_c2f14a15-357d-45a7-84f8-5a4d10b369c4/dns/0.log"
Sep 14 08:30:27.030954 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.030927    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9tqc4_c2f14a15-357d-45a7-84f8-5a4d10b369c4/kube-rbac-proxy/0.log"
Sep 14 08:30:27.116088 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.116060    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ptgdj_378da416-3160-4554-99c3-9f79d9b34aab/dns-node-resolver/0.log"
Sep 14 08:30:27.330915 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.330819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn" event={"ID":"07ef37a7-d38f-4e64-a956-6514c9b4426d","Type":"ContainerStarted","Data":"04a1cf5a23a2b9ccb30248527650228ff866697e4282fe82c7cf76c518f21d1e"}
Sep 14 08:30:27.330915 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.330881    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn" event={"ID":"07ef37a7-d38f-4e64-a956-6514c9b4426d","Type":"ContainerStarted","Data":"9daaf4481c976a8e155526191df48b987d579a758dd3fbf91a07170ee2a038e5"}
Sep 14 08:30:27.331238 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.330988    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:27.347793 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.347737    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn" podStartSLOduration=2.347717673 podStartE2EDuration="2.347717673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 08:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:30:27.344754001 +0000 UTC m=+2483.844146804" watchObservedRunningTime="2026-09-14 08:30:27.347717673 +0000 UTC m=+2483.847110452"
Sep 14 08:30:27.739242 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:27.739213    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wpgcg_618ca54d-af1f-4f80-a9e9-e93710b99d5b/node-ca/0.log"
Sep 14 08:30:29.132606 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:29.132568    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gnjdc_e86a2a63-e727-45a1-bb6b-4e11e821e409/payload-pre-processing/0.log"
Sep 14 08:30:29.165712 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:29.165671    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q6hxd_1231c9fa-d159-43fd-8c3a-024a981babde/payload-processing/0.log"
Sep 14 08:30:30.080844 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:30.080813    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f2vbf_75598ef5-752e-4665-a34f-0e232ee45d96/serve-healthcheck-canary/0.log"
Sep 14 08:30:30.651616 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:30.651586    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4nldc_1c530033-9200-49a2-b490-6fce8c932216/kube-rbac-proxy/0.log"
Sep 14 08:30:30.669804 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:30.669779    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4nldc_1c530033-9200-49a2-b490-6fce8c932216/exporter/0.log"
Sep 14 08:30:30.688675 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:30.688654    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4nldc_1c530033-9200-49a2-b490-6fce8c932216/extractor/0.log"
Sep 14 08:30:32.841802 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:32.841752    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-td4m4_ed06b2ab-ee0b-43d2-8308-b01b4eb31dce/manager/0.log"
Sep 14 08:30:33.145019 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:33.144918    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c8944c479-fflxt_ac0e195d-f177-42ba-97f8-9e88833a15ff/manager/0.log"
Sep 14 08:30:33.164115 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:33.164090    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wkflq_054dc377-2db5-4561-ada1-42c434bc070e/server/0.log"
Sep 14 08:30:33.268054 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:33.268021    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g6z28_9662987c-4423-4a86-b32e-5e6b726d2d3a/manager/0.log"
Sep 14 08:30:33.348748 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:33.348678    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-qlfwn"
Sep 14 08:30:33.423412 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:33.423381    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-ttn8h_308860a3-0a59-4ba6-8c96-d52b471e2810/manager/0.log"
Sep 14 08:30:33.442174 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:33.442146    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-ttn8h_308860a3-0a59-4ba6-8c96-d52b471e2810/copy-manifests/0.log"
Sep 14 08:30:34.699520 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:34.699490    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b895dd7c5-qkcs7_0ae4c6db-6d53-4212-9e84-0e596d105695/manager/0.log"
Sep 14 08:30:36.545383 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:36.545350    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gnjdc_e86a2a63-e727-45a1-bb6b-4e11e821e409/payload-pre-processing/0.log"
Sep 14 08:30:36.577378 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:36.577339    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q6hxd_1231c9fa-d159-43fd-8c3a-024a981babde/payload-processing/0.log"
Sep 14 08:30:38.673497 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:38.673463    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gnjdc_e86a2a63-e727-45a1-bb6b-4e11e821e409/payload-pre-processing/0.log"
Sep 14 08:30:38.705478 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:38.705450    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q6hxd_1231c9fa-d159-43fd-8c3a-024a981babde/payload-processing/0.log"
Sep 14 08:30:40.627867 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:40.627821    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-wsm58_61186fd6-9d8f-43b2-967c-9c1752ced815/kube-storage-version-migrator-operator/1.log"
Sep 14 08:30:40.628925 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:40.628906    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-wsm58_61186fd6-9d8f-43b2-967c-9c1752ced815/kube-storage-version-migrator-operator/0.log"
Sep 14 08:30:41.859464 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.859437    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/kube-multus-additional-cni-plugins/0.log"
Sep 14 08:30:41.878267 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.878243    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/egress-router-binary-copy/0.log"
Sep 14 08:30:41.896391 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.896372    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/cni-plugins/0.log"
Sep 14 08:30:41.914735 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.914712    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/bond-cni-plugin/0.log"
Sep 14 08:30:41.932635 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.932615    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/routeoverride-cni/0.log"
Sep 14 08:30:41.951786 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.951752    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/whereabouts-cni-bincopy/0.log"
Sep 14 08:30:41.983721 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:41.983700    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hm6hk_5fb50d7f-b3c1-4169-a405-ece674e43bfc/whereabouts-cni/0.log"
Sep 14 08:30:42.032508 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:42.032484    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jn2x4_72b04ab9-53df-4a0d-bf76-bc4556235b0c/kube-multus/0.log"
Sep 14 08:30:42.174113 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:42.174088    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k7dvq_be3af394-a1c5-4410-be29-eff8e1a01aa5/network-metrics-daemon/0.log"
Sep 14 08:30:42.190452 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:42.190428    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k7dvq_be3af394-a1c5-4410-be29-eff8e1a01aa5/kube-rbac-proxy/0.log"
Sep 14 08:30:43.025859 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.025753    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/ovn-controller/0.log"
Sep 14 08:30:43.045552 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.045522    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/ovn-acl-logging/0.log"
Sep 14 08:30:43.067495 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.067464    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/kube-rbac-proxy-node/0.log"
Sep 14 08:30:43.089843 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.089813    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/kube-rbac-proxy-ovn-metrics/0.log"
Sep 14 08:30:43.110153 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.110126    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/northd/0.log"
Sep 14 08:30:43.129912 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.129884    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/nbdb/0.log"
Sep 14 08:30:43.149697 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.149663    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/sbdb/0.log"
Sep 14 08:30:43.220385 ip-10-0-131-236 kubenswrapper[2556]: I0914 08:30:43.220353    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cth69_38caa194-e116-44a2-a0d9-294f8841b35d/ovnkube-controller/0.log"