Sep 04 21:53:44.795597 ip-10-0-133-119 systemd[1]: Starting Kubernetes Kubelet...
Sep 04 21:53:45.194702 ip-10-0-133-119 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 21:53:45.194702 ip-10-0-133-119 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 04 21:53:45.194702 ip-10-0-133-119 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 21:53:45.194702 ip-10-0-133-119 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 04 21:53:45.194702 ip-10-0-133-119 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 21:53:45.195434 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.194769    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 04 21:53:45.200267 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200247    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:45.200267 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200264    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:45.200267 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200267    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:45.200267 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200271    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:45.200267 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200274    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200277    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200279    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200282    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200284    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200289    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200292    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200295    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200298    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200301    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200303    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200306    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200308    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200311    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200313    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200316    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200318    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200321    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200325    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:45.200482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200328    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200331    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200333    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200351    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200353    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200356    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200359    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200362    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200365    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200367    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200370    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200372    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200375    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200378    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200380    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200384    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200387    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200389    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200392    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200394    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:45.200934 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200397    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200400    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200402    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200405    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200407    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200409    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200414    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200422    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200426    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200428    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200431    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200433    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200436    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200438    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200441    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200443    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200446    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200448    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200450    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200453    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:45.201429 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200455    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200457    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200460    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200462    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200465    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200468    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200470    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200474    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200477    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200480    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200483    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200485    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200488    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200491    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200493    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200496    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200498    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200501    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200503    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200506    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:45.201905 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200513    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200516    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200518    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200521    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200524    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200527    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200529    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200531    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200534    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200537    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200539    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200541    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200544    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200546    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200548    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.200551    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201223    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201229    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201232    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:45.202431 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201235    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201238    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201241    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201244    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201246    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201249    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201252    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201254    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201257    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201261    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201265    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201269    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201272    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201275    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201278    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201281    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201284    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201286    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201289    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:45.202981 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201294    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201297    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201300    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201303    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201306    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201308    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201311    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201314    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201316    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201319    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201321    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201324    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201327    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201330    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201333    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201351    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201354    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201357    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201360    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201362    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:45.203500 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201365    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201367    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201370    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201373    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201375    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201378    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201380    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201383    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201385    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201388    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201391    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201393    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201396    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201398    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201401    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201403    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201406    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201408    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201411    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201413    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:45.203989 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201416    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201419    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201422    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201425    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201427    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201431    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201433    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201436    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201439    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201441    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201444    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201446    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201449    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201451    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201453    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201456    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201458    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201460    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201463    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201465    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:45.204482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201468    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201471    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201473    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201476    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201478    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201481    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201484    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201487    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201489    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201492    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201494    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201497    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201499    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201502    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201504    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201508    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201511    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201890    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201894    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201897    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:45.204967 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201900    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201903    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201905    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201908    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201910    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201913    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201916    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201919    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201923    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201927    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201930    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201932    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201935    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201938    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201940    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201943    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201945    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201948    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201951    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:45.205477 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201953    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201956    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201959    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201961    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201964    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201966    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201968    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201971    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201973    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201976    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201979    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201981    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201984    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201987    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201990    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201992    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201995    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.201997    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202000    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202002    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:45.205946 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202005    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202007    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202010    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202012    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202015    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202019    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202023    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202026    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202029    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202032    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202035    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202038    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202040    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202043    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202046    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202049    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202051    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202054    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202057    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202060    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:45.206455 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202062    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202065    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202068    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202070    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202072    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202077    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202081    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202083    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202086    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202089    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202091    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202094    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202096    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202098    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202101    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202103    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202106    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202108    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202110    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:45.206948 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202113    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202115    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202118    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202120    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202123    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202125    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202127    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202130    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202132    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202135    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202137    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202140    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202142    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202144    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202146    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202150    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202152    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202155    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202233    2565 flags.go:64] FLAG: --address="0.0.0.0"
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202265    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 04 21:53:45.207419 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202270    2565 flags.go:64] FLAG: --anonymous-auth="true"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202274    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202278    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202281    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202284    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202288    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202291    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202294    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202297    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202301    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202303    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202306    2565 flags.go:64] FLAG: --cgroup-root=""
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202309    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202312    2565 flags.go:64] FLAG: --client-ca-file=""
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202316    2565 flags.go:64] FLAG: --cloud-provider="external"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202319    2565 flags.go:64] FLAG: --cluster-dns="[]"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202323    2565 flags.go:64] FLAG: --cluster-domain=""
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202326    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202329    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202347    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202351    2565 flags.go:64] FLAG: --container-log-max-files="5"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202355    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202358    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 04 21:53:45.207906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202360    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202364    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202367    2565 flags.go:64] FLAG: --contention-profiling="false"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202369    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202372    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202375    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202378    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202381    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202384    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202387    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202390    2565 flags.go:64] FLAG: --enable-load-reader="false"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202393    2565 flags.go:64] FLAG: --enable-server="true"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202395    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202399    2565 flags.go:64] FLAG: --event-burst="100"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202402    2565 flags.go:64] FLAG: --event-qps="50"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202405    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202408    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202410    2565 flags.go:64] FLAG: --eviction-hard=""
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202414    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202417    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202420    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202423    2565 flags.go:64] FLAG: --eviction-soft=""
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202426    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202429    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202434    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 04 21:53:45.208496 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202437    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202440    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202442    2565 flags.go:64] FLAG: --fail-swap-on="true"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202446    2565 flags.go:64] FLAG: --feature-gates=""
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202450    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202453    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202456    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202459    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202461    2565 flags.go:64] FLAG: --healthz-port="10248"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202464    2565 flags.go:64] FLAG: --help="false"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202467    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202470    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202473    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202476    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202480    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202483    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202485    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202488    2565 flags.go:64] FLAG: --image-service-endpoint=""
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202491    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202494    2565 flags.go:64] FLAG: --kube-api-burst="100"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202496    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202499    2565 flags.go:64] FLAG: --kube-api-qps="50"
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202502    2565 flags.go:64] FLAG: --kube-reserved=""
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202505    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 04 21:53:45.209109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202508    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202510    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202513    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202516    2565 flags.go:64] FLAG: --lock-file=""
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202518    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202521    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202524    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202531    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202534    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202537    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202539    2565 flags.go:64] FLAG: --logging-format="text"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202543    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202546    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202549    2565 flags.go:64] FLAG: --manifest-url=""
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202552    2565 flags.go:64] FLAG: --manifest-url-header=""
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202556    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202559    2565 flags.go:64] FLAG: --max-open-files="1000000"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202562    2565 flags.go:64] FLAG: --max-pods="110"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202565    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202568    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202571    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202574    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202576    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202579    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202582    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 04 21:53:45.209698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202589    2565 flags.go:64] FLAG: --node-status-max-images="50"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202591    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202594    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202597    2565 flags.go:64] FLAG: --pod-cidr=""
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202600    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202605    2565 flags.go:64] FLAG: --pod-manifest-path=""
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202608    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202611    2565 flags.go:64] FLAG: --pods-per-core="0"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202614    2565 flags.go:64] FLAG: --port="10250"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202617    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202620    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eab0c388d2a28104"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202622    2565 flags.go:64] FLAG: --qos-reserved=""
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202625    2565 flags.go:64] FLAG: --read-only-port="10255"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202628    2565 flags.go:64] FLAG: --register-node="true"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202632    2565 flags.go:64] FLAG: --register-with-taints=""
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202635    2565 flags.go:64] FLAG: --registry-burst="10"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202639    2565 flags.go:64] FLAG: --registry-qps="5"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202642    2565 flags.go:64] FLAG: --reserved-cpus=""
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202644    2565 flags.go:64] FLAG: --reserved-memory=""
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202649    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202651    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202654    2565 flags.go:64] FLAG: --rotate-certificates="false"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202657    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202660    2565 flags.go:64] FLAG: --runonce="false"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202662    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 04 21:53:45.210326 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202665    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202668    2565 flags.go:64] FLAG: --seccomp-default="false"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202671    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202673    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202676    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202679    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202682    2565 flags.go:64] FLAG: --storage-driver-password="root"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202685    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202688    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202690    2565 flags.go:64] FLAG: --storage-driver-user="root"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202693    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202696    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202699    2565 flags.go:64] FLAG: --system-cgroups=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202701    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202706    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202709    2565 flags.go:64] FLAG: --tls-cert-file=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202712    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202716    2565 flags.go:64] FLAG: --tls-min-version=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202718    2565 flags.go:64] FLAG: --tls-private-key-file=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202721    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202724    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202728    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202731    2565 flags.go:64] FLAG: --v="2"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202734    2565 flags.go:64] FLAG: --version="false"
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202742    2565 flags.go:64] FLAG: --vmodule=""
Sep 04 21:53:45.210958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202745    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.202750    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202857    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202860    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202863    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202866    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202869    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202872    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202875    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202877    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202880    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202882    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202885    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202887    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202890    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202892    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202894    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202897    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202900    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202903    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202905    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:45.211591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202908    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202910    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202913    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202915    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202918    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202920    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202923    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202927    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202930    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202932    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202935    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202938    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202942    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202945    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202947    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202950    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202952    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202955    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202957    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:45.212135 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202960    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202963    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202965    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202968    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202970    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202973    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202976    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202980    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202983    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202987    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202991    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202994    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.202997    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203000    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203002    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203005    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203007    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203010    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203012    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:45.212628 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203015    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203018    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203021    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203024    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203026    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203029    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203032    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203035    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203038    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203041    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203043    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203046    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203048    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203051    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203053    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203056    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203058    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203061    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203063    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203066    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:45.213089 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203068    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203071    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203073    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203076    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203078    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203081    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203083    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203085    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203088    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203090    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203093    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203096    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203098    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203102    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203104    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203107    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203109    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203112    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203116    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:45.213590 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203120    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203123    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.203126    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.203696    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.212113    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.212129    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.212986    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.212995    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213065    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213071    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213074    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213076    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213080    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213083    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213087    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:45.214057 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213091    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213095    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213098    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213101    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213104    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213107    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213110    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213113    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213116    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213119    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213122    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213125    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213127    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213130    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213132    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213135    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213137    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213140    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213143    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213145    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:45.214461 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213148    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213150    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213154    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213157    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213160    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213163    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213165    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213168    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213170    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213173    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213176    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213178    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213181    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213183    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213186    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213189    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213191    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213194    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213196    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213198    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:45.214942 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213201    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213205    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213208    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213211    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213213    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213216    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213219    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213221    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213224    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213226    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213229    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213231    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213234    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213237    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213239    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213242    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213245    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213247    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:45.215482 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213251    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213254    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213257    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213259    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213262    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213265    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213268    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213270    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213272    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213275    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213277    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213279    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213282    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213284    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213287    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213289    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213292    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213294    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213296    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213298    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:45.215916 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213301    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213303    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213306    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213308    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213311    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213313    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213315    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213318    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213321    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213323    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213326    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213328    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213331    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213333    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.213350    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 21:53:45.216472 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213457    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213463    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213465    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213468    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213471    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213474    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213477    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213480    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213483    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213485    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213488    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213491    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213493    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213496    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213498    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213501    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213503    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213506    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213508    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:45.216849 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213511    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213513    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213516    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213518    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213521    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213524    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213526    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213529    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213531    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213534    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213537    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213539    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213541    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213544    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213546    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213548    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213551    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213553    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213556    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213558    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:45.217286 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213560    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213563    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213565    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213568    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213570    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213573    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213575    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213578    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213580    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213582    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213585    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213587    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213590    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213592    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213595    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213597    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213600    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213602    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213605    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213607    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:45.217773 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213610    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213612    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213615    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213618    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213620    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213622    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213625    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213627    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213630    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213633    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213636    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213638    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213641    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213643    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213645    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213648    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213651    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213655    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213658    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:45.218257 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213660    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213663    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213665    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213668    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213671    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213673    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213676    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213678    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213680    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213683    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213685    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213687    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213691    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213694    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213697    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213700    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213703    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213705    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213708    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213710    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:45.218722 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:45.213712    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:45.219195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.213717    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 21:53:45.219195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.213821    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 04 21:53:45.219195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.216434    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 04 21:53:45.219195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.217214    2565 server.go:1019] "Starting client certificate rotation"
Sep 04 21:53:45.219195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.217313    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 21:53:45.219195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.217354    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 21:53:45.219666 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.219654    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 21:53:45.220777 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.220759    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 21:53:45.229883 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.229866    2565 log.go:25] "Validated CRI v1 runtime API"
Sep 04 21:53:45.232683 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.232668    2565 log.go:25] "Validated CRI v1 image API"
Sep 04 21:53:45.233591 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.233573    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 04 21:53:45.238654 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.238635    2565 fs.go:135] Filesystem UUIDs: map[63788504-68ff-408a-ade3-2144d4de84b0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 959c9a6e-4383-4f44-aa30-30076062386d:/dev/nvme0n1p4]
Sep 04 21:53:45.238726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.238652    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 04 21:53:45.244497 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.244377    2565 manager.go:216] Machine: {Timestamp:2026-09-04 21:53:45.243285119 +0000 UTC m=+0.327669955 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107854 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ab11f599f8186b32074c2770a5bf1 SystemUUID:ec2ab11f-599f-8186-b320-74c2770a5bf1 BootID:a37f9a73-8925-4768-b960-eefd7ddfb556 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c2:8e:ef:ed:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c2:8e:ef:ed:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:79:93:c0:e6:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 04 21:53:45.244497 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.244490    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 04 21:53:45.244615 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.244566    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 04 21:53:45.247404 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.247379    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 04 21:53:45.247540 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.247407    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-119.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 04 21:53:45.248139 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.248129    2565 topology_manager.go:138] "Creating topology manager with none policy"
Sep 04 21:53:45.248168 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.248142    2565 container_manager_linux.go:309] "Creating device plugin manager"
Sep 04 21:53:45.248168 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.248150    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 21:53:45.248799 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.248788    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 21:53:45.250512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.250502    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 04 21:53:45.252276 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.252260    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 21:53:45.254450 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254438    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 21:53:45.254555 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254546    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 04 21:53:45.254603 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254581    2565 kubelet.go:486] "Attempting to sync node with API server"
Sep 04 21:53:45.254603 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254598    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 04 21:53:45.254688 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254616    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 04 21:53:45.254688 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254627    2565 kubelet.go:398] "Adding apiserver pod source"
Sep 04 21:53:45.254688 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254636    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 04 21:53:45.254978 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254966    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 21:53:45.255024 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.254985    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 21:53:45.257604 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.257590    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 04 21:53:45.259724 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.259708    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 04 21:53:45.259799 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.259749    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 04 21:53:45.260472 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260459    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260480    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260498    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260506    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260518    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260526    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260546    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260553    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 04 21:53:45.260564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260562    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 04 21:53:45.260821 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260587    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 04 21:53:45.260821 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260605    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 04 21:53:45.260821 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.260614    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 04 21:53:45.261555 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.261543    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 04 21:53:45.261605 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.261558    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 04 21:53:45.264714 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.264691    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-119.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 04 21:53:45.264911 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.264890    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 04 21:53:45.264975 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.264918    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 04 21:53:45.265512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.265500    2565 server.go:1268] "Started kubelet"
Sep 04 21:53:45.265758 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.265713    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 04 21:53:45.266055 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.265856    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 04 21:53:45.266208 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.266191    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 04 21:53:45.266380 ip-10-0-133-119 systemd[1]: Started Kubernetes Kubelet.
Sep 04 21:53:45.268131 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.268098    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 04 21:53:45.272867 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.272741    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 04 21:53:45.272959 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.272829    2565 server.go:310] "Adding debug handlers to kubelet server"
Sep 04 21:53:45.274569 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.274553    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 04 21:53:45.275029 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275006    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 04 21:53:45.275138 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.275118    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.275649 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275632    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 04 21:53:45.275649 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275635    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 04 21:53:45.275763 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275660    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 04 21:53:45.275763 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275744    2565 reconstruct.go:97] "Volume reconstruction finished"
Sep 04 21:53:45.275763 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275754    2565 reconciler.go:29] "Reconciler: start to sync state"
Sep 04 21:53:45.275939 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275926    2565 factory.go:55] Registering systemd factory
Sep 04 21:53:45.275939 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.275941    2565 factory.go:223] Registration of the systemd container factory successfully
Sep 04 21:53:45.276132 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.276119    2565 factory.go:153] Registering CRI-O factory
Sep 04 21:53:45.276178 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.276134    2565 factory.go:223] Registration of the crio container factory successfully
Sep 04 21:53:45.276178 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.276173    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 04 21:53:45.276242 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.276185    2565 factory.go:103] Registering Raw factory
Sep 04 21:53:45.276242 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.276196    2565 manager.go:1196] Started watching for new ooms in manager
Sep 04 21:53:45.277100 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.277071    2565 manager.go:349] Starting recovery of all containers
Sep 04 21:53:45.278278 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.278250    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 04 21:53:45.278445 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.278423    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 04 21:53:45.279409 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.278409    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-119.ec2.internal.18d23cc67fd6a21d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-119.ec2.internal,UID:ip-10-0-133-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-119.ec2.internal,},FirstTimestamp:2026-09-04 21:53:45.265472029 +0000 UTC m=+0.349856864,LastTimestamp:2026-09-04 21:53:45.265472029 +0000 UTC m=+0.349856864,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-119.ec2.internal,}"
Sep 04 21:53:45.280680 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.280654    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mxvsl"
Sep 04 21:53:45.287481 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.287462    2565 manager.go:354] Recovery completed
Sep 04 21:53:45.288766 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.288741    2565 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 04 21:53:45.291157 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.291133    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mxvsl"
Sep 04 21:53:45.292097 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.292075    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.294019 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294001    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.294090 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294033    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.294090 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294044    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.294570 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294556    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 04 21:53:45.294570 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294563    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.294570 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294570    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 04 21:53:45.294694 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.294587    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 21:53:45.295226 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.295211    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.295292 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.295235    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.295292 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.295244    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.296118 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.296052    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-119.ec2.internal.18d23cc6818a34e3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-119.ec2.internal,UID:ip-10-0-133-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-119.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-119.ec2.internal,},FirstTimestamp:2026-09-04 21:53:45.294017763 +0000 UTC m=+0.378402600,LastTimestamp:2026-09-04 21:53:45.294017763 +0000 UTC m=+0.378402600,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-119.ec2.internal,}"
Sep 04 21:53:45.296593 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.296582    2565 policy_none.go:49] "None policy: Start"
Sep 04 21:53:45.296629 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.296597    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 04 21:53:45.296629 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.296607    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 04 21:53:45.298621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.298611    2565 policy_none.go:47] "Start"
Sep 04 21:53:45.359117 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.358901    2565 manager.go:337] "Starting Device Plugin manager"
Sep 04 21:53:45.359117 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.358943    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 04 21:53:45.359117 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.358957    2565 server.go:87] "Starting device plugin registration server"
Sep 04 21:53:45.359257 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.359240    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 04 21:53:45.359289 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.359252    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 04 21:53:45.359393 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.359372    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 04 21:53:45.359550 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.359450    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 04 21:53:45.359550 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.359457    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 04 21:53:45.359916 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.359894    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 04 21:53:45.359991 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.359954    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.382290 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.382264    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 04 21:53:45.383607 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.383592    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 04 21:53:45.383669 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.383646    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 04 21:53:45.383715 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.383671    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 04 21:53:45.383766 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.383713    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 04 21:53:45.385919 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.385903    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:45.460161 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.460052    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.461079 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.461056    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.461201 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.461093    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.461201 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.461110    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.461201 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.461135    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.470384 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.470363    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.470443 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.470388    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-119.ec2.internal\": node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.483906 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.483880    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"]
Sep 04 21:53:45.484009 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.483941    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.485516 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.485499    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.485598 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.485528    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.485598 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.485538    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.487732 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.487719    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.487846 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.487828    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.487889 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.487855    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.488515 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.488493    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.488596 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.488522    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.488596 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.488537    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.488596 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.488494    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.488741 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.488598    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.488741 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.488612    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.490895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.490879    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.490978 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.490907    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.491205 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.491187    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.491690 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.491673    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.491770 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.491696    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.491770 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.491705    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.507859 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.507840    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-119.ec2.internal\" not found" node="ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.511239 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.511225    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-119.ec2.internal\" not found" node="ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.591550 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.591523    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.677397 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.677372    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85ce9a2a03055b1e4010209d2f26585d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-119.ec2.internal\" (UID: \"85ce9a2a03055b1e4010209d2f26585d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.677514 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.677405    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e4eb6a818c7d4b42d870eb4178268581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal\" (UID: \"e4eb6a818c7d4b42d870eb4178268581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.677514 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.677423    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4eb6a818c7d4b42d870eb4178268581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal\" (UID: \"e4eb6a818c7d4b42d870eb4178268581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.692459 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.692433    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.778123 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.778056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e4eb6a818c7d4b42d870eb4178268581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal\" (UID: \"e4eb6a818c7d4b42d870eb4178268581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.778123 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.778093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4eb6a818c7d4b42d870eb4178268581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal\" (UID: \"e4eb6a818c7d4b42d870eb4178268581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.778123 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.778112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85ce9a2a03055b1e4010209d2f26585d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-119.ec2.internal\" (UID: \"85ce9a2a03055b1e4010209d2f26585d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.778289 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.778166    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4eb6a818c7d4b42d870eb4178268581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal\" (UID: \"e4eb6a818c7d4b42d870eb4178268581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.778289 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.778168    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e4eb6a818c7d4b42d870eb4178268581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal\" (UID: \"e4eb6a818c7d4b42d870eb4178268581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.778289 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.778221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85ce9a2a03055b1e4010209d2f26585d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-119.ec2.internal\" (UID: \"85ce9a2a03055b1e4010209d2f26585d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.793163 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.793145    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.810320 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.810298    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.813966 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:45.813950    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:45.893984 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.893959    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:45.994419 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:45.994398    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:46.094971 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.094914    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-119.ec2.internal\" not found"
Sep 04 21:53:46.194651 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.194628    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:46.216871 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.216852    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 04 21:53:46.217260 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.216960    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 04 21:53:46.217260 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.216987    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 04 21:53:46.255177 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.255153    2565 apiserver.go:52] "Watching apiserver"
Sep 04 21:53:46.270768 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.270745    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 04 21:53:46.271074 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.271053    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vrw2k","openshift-multus/multus-pfzcr","openshift-network-operator/iptables-alerter-nhv96","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc","openshift-dns/node-resolver-pj2xr","openshift-multus/network-metrics-daemon-8nmrf","openshift-network-diagnostics/network-check-target-x6rqt","openshift-ovn-kubernetes/ovnkube-node-5cffb","kube-system/global-pull-secret-syncer-f96gh","kube-system/konnectivity-agent-pmzjn","openshift-cluster-node-tuning-operator/tuned-cz75q","openshift-image-registry/node-ca-llxl2"]
Sep 04 21:53:46.274594 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.274573    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.274679 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.274658    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 04 21:53:46.276040 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.276024    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:46.276533 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.276518    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.276857 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.276840    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 04 21:53:46.276943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.276862    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.276999 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.276949    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 04 21:53:46.277050 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.277025    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xjxsh\""
Sep 04 21:53:46.277100 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.277080    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 04 21:53:46.277146 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.277133    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.279435 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.279421    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.280322 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280307    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 04 21:53:46.280432 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280416    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jgjll\""
Sep 04 21:53:46.280494 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280477    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-cnibin\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.280536 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280504    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.280570 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280542    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-system-cni-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280612 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-cnibin\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280612 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-cni-multus\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280612 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280602    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-conf-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280713 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280620    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-system-cni-dir\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.280713 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.280713 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-netns\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280713 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280697    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-kubelet\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280713 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280712    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-hostroot\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280726    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsw44\" (UniqueName: \"kubernetes.io/projected/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-kube-api-access-tsw44\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280762    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-cni-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-cni-binary-copy\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280848    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-socket-dir-parent\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280872    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-k8s-cni-cncf-io\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-multus-certs\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.280943 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280917    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.281239 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmt6v\" (UniqueName: \"kubernetes.io/projected/dad701d3-4293-4879-9296-70fcfc8100fd-kube-api-access-pmt6v\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.281239 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-cni-bin\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.281239 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.280990    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-daemon-config\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.281239 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.281011    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-etc-kubernetes\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.281454 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.281385    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.281527 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.281512    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qr8nv\""
Sep 04 21:53:46.281662 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.281641    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 04 21:53:46.281731 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.281642    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.281834 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.281767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.283737 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.283721    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.284829 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.284815    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.284927 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.284911    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.284984 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.284926    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j5kcv\""
Sep 04 21:53:46.285183 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.285169    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 04 21:53:46.285559 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.285548    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.285854 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.285839    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.285965 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.285947    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.286073 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.285978    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hl8jm\""
Sep 04 21:53:46.286073 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.286010    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:46.288718 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.288700    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:46.288805 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.288777    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:46.289105 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.289082    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 21:53:46.289234 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.289167    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"
Sep 04 21:53:46.290802 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.290786    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 21:53:46.291254 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.291237    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.293299 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293281    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 04 21:53:46.293436 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293327    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 04 21:53:46.293511 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293454    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.293578 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293525    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 04 21:53:46.293578 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293538    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 04 21:53:46.293578 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293556    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x75w5\""
Sep 04 21:53:46.293779 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293766    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.293816 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.293792    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.293852 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.293822    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:46.294835 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.294814    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-03 21:48:45 +0000 UTC" deadline="2028-05-08 22:56:27.144256104 +0000 UTC"
Sep 04 21:53:46.294881 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.294835    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14689h2m40.849423541s"
Sep 04 21:53:46.296373 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.296359    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.299398 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.298540    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gkdzt\""
Sep 04 21:53:46.299398 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.298679    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 04 21:53:46.299398 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.298733    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.299398 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.298838    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 04 21:53:46.299754 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.299737    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 21:53:46.301396 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.301144    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vhvnl\""
Sep 04 21:53:46.301479 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.301463    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.302245 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.302227    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal"]
Sep 04 21:53:46.302317 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.302264    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal"]
Sep 04 21:53:46.302383 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.302320    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.302861 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.302843    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.304228 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.304182    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 04 21:53:46.304228 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.304218    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-slr9c\""
Sep 04 21:53:46.304410 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.304396    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 04 21:53:46.304477 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.304397    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 04 21:53:46.325091 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.325070    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9cwws"
Sep 04 21:53:46.333996 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.333969    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9cwws"
Sep 04 21:53:46.351525 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.351469    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4eb6a818c7d4b42d870eb4178268581.slice/crio-12391136e4821fd8f07ab5323cbaf63b4dd079d20dec5bab08253a4b0d7ed999 WatchSource:0}: Error finding container 12391136e4821fd8f07ab5323cbaf63b4dd079d20dec5bab08253a4b0d7ed999: Status 404 returned error can't find the container with id 12391136e4821fd8f07ab5323cbaf63b4dd079d20dec5bab08253a4b0d7ed999
Sep 04 21:53:46.351698 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.351685    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ce9a2a03055b1e4010209d2f26585d.slice/crio-0c0587e75df9d17a0a7181f8001eac8e74f36943b279fb31a615458ae95645c6 WatchSource:0}: Error finding container 0c0587e75df9d17a0a7181f8001eac8e74f36943b279fb31a615458ae95645c6: Status 404 returned error can't find the container with id 0c0587e75df9d17a0a7181f8001eac8e74f36943b279fb31a615458ae95645c6
Sep 04 21:53:46.357715 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.357700    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 21:53:46.376592 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.376510    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 04 21:53:46.381162 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-hostroot\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381238 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfg4v\" (UniqueName: \"kubernetes.io/projected/36f2571c-c2b3-4884-bb88-c5182fdaffc5-kube-api-access-sfg4v\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.381238 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-node-log\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.381238 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381225    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovnkube-script-lib\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.381382 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381257    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysctl-conf\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.381382 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381288    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-cni-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381382 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381250    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-hostroot\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381382 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36f2571c-c2b3-4884-bb88-c5182fdaffc5-iptables-alerter-script\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.381382 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381380    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-cni-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-slash\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysconfig\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-lib-modules\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381532    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-daemon-config\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381582    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.381621 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381605    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-sys\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c768d1b-c85f-459b-b512-048becacaf06-etc-tuned\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovn-node-metrics-cert\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-kubelet\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-kubelet\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381805    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-systemd-units\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.381874 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381848    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysctl-d\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-cni-binary-copy\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-socket-dir-parent\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381928    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-multus-certs\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5376eed9-b670-4a76-8ee4-bca13d83d09c-host\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381958    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-daemon-config\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.381997    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-socket-dir-parent\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382010    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-multus-certs\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382028    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-modprobe-d\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382065    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-cnibin\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382087    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-conf-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382116    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjtb\" (UniqueName: \"kubernetes.io/projected/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-kube-api-access-7pjtb\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-cnibin\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382143    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266ddbfd-33be-4063-8b22-694081563177-tmp-dir\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382165    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lf7h\" (UniqueName: \"kubernetes.io/projected/453903df-4411-410d-a1b5-9a513da936c4-kube-api-access-9lf7h\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382177    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-multus-conf-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382189    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-kubelet-config\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-var-lib-kubelet\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-system-cni-dir\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382284    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-netns\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382355    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-netns\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382365    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-kubelet\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382330    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-system-cni-dir\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382403    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5376eed9-b670-4a76-8ee4-bca13d83d09c-serviceca\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382412    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-cni-binary-copy\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc7pg\" (UniqueName: \"kubernetes.io/projected/266ddbfd-33be-4063-8b22-694081563177-kube-api-access-dc7pg\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.382491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-systemd\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-k8s-cni-cncf-io\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382494    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-registration-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382507    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-log-socket\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-etc-kubernetes\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382536    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36f2571c-c2b3-4884-bb88-c5182fdaffc5-host-slash\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382531    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-run-k8s-cni-cncf-io\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382552    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382567    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrrsr\" (UniqueName: \"kubernetes.io/projected/5376eed9-b670-4a76-8ee4-bca13d83d09c-kube-api-access-jrrsr\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382581    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j6xl\" (UniqueName: \"kubernetes.io/projected/7c768d1b-c85f-459b-b512-048becacaf06-kube-api-access-9j6xl\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382580    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-etc-kubernetes\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382594    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-cnibin\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382625    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-system-cni-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382643    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-cnibin\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-cni-multus\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-cni-multus\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382694    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-etc-selinux\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.383264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-system-cni-dir\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382751    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-cni-netd\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382765    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/266ddbfd-33be-4063-8b22-694081563177-hosts-file\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382781    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-host\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-etc-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382823    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382858    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-cni-bin\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-env-overrides\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382916    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.382944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsw44\" (UniqueName: \"kubernetes.io/projected/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-kube-api-access-tsw44\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-run-netns\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383065    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383080    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-var-lib-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b8f1fce-1378-4ae4-9b93-bbde1a048fa6-agent-certs\") pod \"konnectivity-agent-pmzjn\" (UID: \"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6\") " pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.383825 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383111    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b8f1fce-1378-4ae4-9b93-bbde1a048fa6-konnectivity-ca\") pod \"konnectivity-agent-pmzjn\" (UID: \"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6\") " pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-dbus\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383146    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-run\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmt6v\" (UniqueName: \"kubernetes.io/projected/dad701d3-4293-4879-9296-70fcfc8100fd-kube-api-access-pmt6v\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-cni-bin\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383204    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-host-var-lib-cni-bin\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-socket-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383243    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-sys-fs\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383245    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dad701d3-4293-4879-9296-70fcfc8100fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383265    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-ovn\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383288    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovnkube-config\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5m94\" (UniqueName: \"kubernetes.io/projected/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-kube-api-access-g5m94\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383326    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-kubernetes\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383387    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-device-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.384252 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383452    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-systemd\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.384847 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c768d1b-c85f-459b-b512-048becacaf06-tmp\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.384847 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.383752    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dad701d3-4293-4879-9296-70fcfc8100fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.386462 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.386427    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal" event={"ID":"e4eb6a818c7d4b42d870eb4178268581","Type":"ContainerStarted","Data":"12391136e4821fd8f07ab5323cbaf63b4dd079d20dec5bab08253a4b0d7ed999"}
Sep 04 21:53:46.387353 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.387325    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal" event={"ID":"85ce9a2a03055b1e4010209d2f26585d","Type":"ContainerStarted","Data":"0c0587e75df9d17a0a7181f8001eac8e74f36943b279fb31a615458ae95645c6"}
Sep 04 21:53:46.390833 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.390817    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 04 21:53:46.394401 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.394271    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmt6v\" (UniqueName: \"kubernetes.io/projected/dad701d3-4293-4879-9296-70fcfc8100fd-kube-api-access-pmt6v\") pod \"multus-additional-cni-plugins-vrw2k\" (UID: \"dad701d3-4293-4879-9296-70fcfc8100fd\") " pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.394473 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.394373    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsw44\" (UniqueName: \"kubernetes.io/projected/7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5-kube-api-access-tsw44\") pod \"multus-pfzcr\" (UID: \"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5\") " pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.464387 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.464364    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:46.484396 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484371    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/266ddbfd-33be-4063-8b22-694081563177-hosts-file\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-host\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484426    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-etc-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/266ddbfd-33be-4063-8b22-694081563177-hosts-file\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484470    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-cni-bin\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484479    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-host\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.484499 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-env-overrides\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484502    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-etc-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484524    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-cni-bin\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484566    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484568    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-run-netns\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-run-netns\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-var-lib-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b8f1fce-1378-4ae4-9b93-bbde1a048fa6-agent-certs\") pod \"konnectivity-agent-pmzjn\" (UID: \"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6\") " pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484658    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-var-lib-openvswitch\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b8f1fce-1378-4ae4-9b93-bbde1a048fa6-konnectivity-ca\") pod \"konnectivity-agent-pmzjn\" (UID: \"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6\") " pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484704    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-dbus\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484726    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-run\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-socket-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-sys-fs\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.484860 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484837    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-sys-fs\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484885    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-dbus\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-run\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484942    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-ovn\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484949    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-socket-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484955    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-env-overrides\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.484984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovnkube-config\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485011    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5m94\" (UniqueName: \"kubernetes.io/projected/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-kube-api-access-g5m94\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485017    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-ovn\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-kubernetes\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-device-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485082    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-systemd\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485104    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c768d1b-c85f-459b-b512-048becacaf06-tmp\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485117    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-kubernetes\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfg4v\" (UniqueName: \"kubernetes.io/projected/36f2571c-c2b3-4884-bb88-c5182fdaffc5-kube-api-access-sfg4v\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485152    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-node-log\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485174    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovnkube-script-lib\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysctl-conf\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.485663 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b8f1fce-1378-4ae4-9b93-bbde1a048fa6-konnectivity-ca\") pod \"konnectivity-agent-pmzjn\" (UID: \"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6\") " pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36f2571c-c2b3-4884-bb88-c5182fdaffc5-iptables-alerter-script\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-slash\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485253    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-run-systemd\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485265    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysconfig\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485286    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-node-log\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-lib-modules\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485316    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485356    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-sys\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485386    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-lib-modules\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485174    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-device-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c768d1b-c85f-459b-b512-048becacaf06-etc-tuned\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovn-node-metrics-cert\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-systemd-units\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485467    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485490    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysctl-d\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.486580 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485512    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovnkube-config\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485521    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysctl-conf\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5376eed9-b670-4a76-8ee4-bca13d83d09c-host\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485544    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-slash\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485570    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-modprobe-d\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485580    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485582    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysconfig\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485588    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-systemd-units\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485600    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjtb\" (UniqueName: \"kubernetes.io/projected/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-kube-api-access-7pjtb\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485625    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266ddbfd-33be-4063-8b22-694081563177-tmp-dir\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485650    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lf7h\" (UniqueName: \"kubernetes.io/projected/453903df-4411-410d-a1b5-9a513da936c4-kube-api-access-9lf7h\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.485655    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485645    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5376eed9-b670-4a76-8ee4-bca13d83d09c-host\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-kubelet-config\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485699    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-var-lib-kubelet\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-sys\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.485761    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:46.985719806 +0000 UTC m=+2.070104634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.485762    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:46.487118 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485778    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-kubelet\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5376eed9-b670-4a76-8ee4-bca13d83d09c-serviceca\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.485822    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:46.985804924 +0000 UTC m=+2.070189766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc7pg\" (UniqueName: \"kubernetes.io/projected/266ddbfd-33be-4063-8b22-694081563177-kube-api-access-dc7pg\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36f2571c-c2b3-4884-bb88-c5182fdaffc5-iptables-alerter-script\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-systemd\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485928    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-registration-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-log-socket\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485973    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-sysctl-d\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.485983    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36f2571c-c2b3-4884-bb88-c5182fdaffc5-host-slash\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486009    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486016    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-systemd\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrrsr\" (UniqueName: \"kubernetes.io/projected/5376eed9-b670-4a76-8ee4-bca13d83d09c-kube-api-access-jrrsr\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486055    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6xl\" (UniqueName: \"kubernetes.io/projected/7c768d1b-c85f-459b-b512-048becacaf06-kube-api-access-9j6xl\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-etc-selinux\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486104    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.487693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-cni-netd\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5376eed9-b670-4a76-8ee4-bca13d83d09c-serviceca\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486190    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-cni-netd\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486241    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-registration-dir\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486250    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-etc-modprobe-d\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486268    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-kubelet-config\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486284    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-log-socket\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486308    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-kubelet\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486321    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36f2571c-c2b3-4884-bb88-c5182fdaffc5-host-slash\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486300    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c768d1b-c85f-459b-b512-048becacaf06-var-lib-kubelet\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486388    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-etc-selinux\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486506    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/266ddbfd-33be-4063-8b22-694081563177-tmp-dir\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.486909    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovnkube-script-lib\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.488182 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.487668    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b8f1fce-1378-4ae4-9b93-bbde1a048fa6-agent-certs\") pod \"konnectivity-agent-pmzjn\" (UID: \"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6\") " pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.488693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.488263    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c768d1b-c85f-459b-b512-048becacaf06-etc-tuned\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.488693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.488614    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c768d1b-c85f-459b-b512-048becacaf06-tmp\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.488755 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.488735    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-ovn-node-metrics-cert\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.495780 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.495762    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5m94\" (UniqueName: \"kubernetes.io/projected/73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71-kube-api-access-g5m94\") pod \"ovnkube-node-5cffb\" (UID: \"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.496060 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.496045    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfg4v\" (UniqueName: \"kubernetes.io/projected/36f2571c-c2b3-4884-bb88-c5182fdaffc5-kube-api-access-sfg4v\") pod \"iptables-alerter-nhv96\" (UID: \"36f2571c-c2b3-4884-bb88-c5182fdaffc5\") " pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.496792 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.496775    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:46.496792 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.496791    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:46.496924 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.496800    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:46.496924 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.496842    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:46.996832377 +0000 UTC m=+2.081217225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:46.497986 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.497968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc7pg\" (UniqueName: \"kubernetes.io/projected/266ddbfd-33be-4063-8b22-694081563177-kube-api-access-dc7pg\") pod \"node-resolver-pj2xr\" (UID: \"266ddbfd-33be-4063-8b22-694081563177\") " pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.503916 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.503897    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrrsr\" (UniqueName: \"kubernetes.io/projected/5376eed9-b670-4a76-8ee4-bca13d83d09c-kube-api-access-jrrsr\") pod \"node-ca-llxl2\" (UID: \"5376eed9-b670-4a76-8ee4-bca13d83d09c\") " pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.504636 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.504619    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lf7h\" (UniqueName: \"kubernetes.io/projected/453903df-4411-410d-a1b5-9a513da936c4-kube-api-access-9lf7h\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.507150 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.507129    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j6xl\" (UniqueName: \"kubernetes.io/projected/7c768d1b-c85f-459b-b512-048becacaf06-kube-api-access-9j6xl\") pod \"tuned-cz75q\" (UID: \"7c768d1b-c85f-459b-b512-048becacaf06\") " pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.508125 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.508077    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjtb\" (UniqueName: \"kubernetes.io/projected/2ecbaa34-70f9-4014-9d8a-f6392fea9f8e-kube-api-access-7pjtb\") pod \"aws-ebs-csi-driver-node-5f6nc\" (UID: \"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.520126 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.520112    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:46.600739 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.600718    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vrw2k"
Sep 04 21:53:46.610831 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.610809    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad701d3_4293_4879_9296_70fcfc8100fd.slice/crio-69f237f785b07ee1353a44b3b040afc37b5146acf418e299b1d53224b80e9af9 WatchSource:0}: Error finding container 69f237f785b07ee1353a44b3b040afc37b5146acf418e299b1d53224b80e9af9: Status 404 returned error can't find the container with id 69f237f785b07ee1353a44b3b040afc37b5146acf418e299b1d53224b80e9af9
Sep 04 21:53:46.616378 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.616361    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pfzcr"
Sep 04 21:53:46.624247 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.624228    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddff86b_b44f_4dcc_bdbb_a6c56ac06da5.slice/crio-bb67904b3b8f319de84fcdd79e009538072f889e3cd456711cf8c92df32067c4 WatchSource:0}: Error finding container bb67904b3b8f319de84fcdd79e009538072f889e3cd456711cf8c92df32067c4: Status 404 returned error can't find the container with id bb67904b3b8f319de84fcdd79e009538072f889e3cd456711cf8c92df32067c4
Sep 04 21:53:46.630659 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.630644    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nhv96"
Sep 04 21:53:46.634451 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.634435    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc"
Sep 04 21:53:46.638281 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.638262    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f2571c_c2b3_4884_bb88_c5182fdaffc5.slice/crio-776a0127dd5c7d0f64e1e0623a9c66607e86c74a6ff061c3cba674114da4d001 WatchSource:0}: Error finding container 776a0127dd5c7d0f64e1e0623a9c66607e86c74a6ff061c3cba674114da4d001: Status 404 returned error can't find the container with id 776a0127dd5c7d0f64e1e0623a9c66607e86c74a6ff061c3cba674114da4d001
Sep 04 21:53:46.642881 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.642861    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ecbaa34_70f9_4014_9d8a_f6392fea9f8e.slice/crio-9d15180352a5fea190a951d7182205cbcd54686a2099baad75b28c9f37131374 WatchSource:0}: Error finding container 9d15180352a5fea190a951d7182205cbcd54686a2099baad75b28c9f37131374: Status 404 returned error can't find the container with id 9d15180352a5fea190a951d7182205cbcd54686a2099baad75b28c9f37131374
Sep 04 21:53:46.657665 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.657649    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pj2xr"
Sep 04 21:53:46.663186 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.663170    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:53:46.665904 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.665781    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266ddbfd_33be_4063_8b22_694081563177.slice/crio-0e6febf8bad0c8b2561abab8f1ad591af2048b8f029fbc266a78412ef4c50abb WatchSource:0}: Error finding container 0e6febf8bad0c8b2561abab8f1ad591af2048b8f029fbc266a78412ef4c50abb: Status 404 returned error can't find the container with id 0e6febf8bad0c8b2561abab8f1ad591af2048b8f029fbc266a78412ef4c50abb
Sep 04 21:53:46.669420 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.669405    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:53:46.671279 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.671261    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a8e6b9_c75a_4cf9_8a26_5fa6c3ba9a71.slice/crio-e0cadfd750e63b53388b834c8a6e89b9fbcb215609a3f02f765d0daa72d4491d WatchSource:0}: Error finding container e0cadfd750e63b53388b834c8a6e89b9fbcb215609a3f02f765d0daa72d4491d: Status 404 returned error can't find the container with id e0cadfd750e63b53388b834c8a6e89b9fbcb215609a3f02f765d0daa72d4491d
Sep 04 21:53:46.674504 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.674490    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cz75q"
Sep 04 21:53:46.677950 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.677929    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8f1fce_1378_4ae4_9b93_bbde1a048fa6.slice/crio-9bf6c08a7ed1b6743ff14f4ee3836cc7bef0f6b9afcd0078959ed2af42d51c3c WatchSource:0}: Error finding container 9bf6c08a7ed1b6743ff14f4ee3836cc7bef0f6b9afcd0078959ed2af42d51c3c: Status 404 returned error can't find the container with id 9bf6c08a7ed1b6743ff14f4ee3836cc7bef0f6b9afcd0078959ed2af42d51c3c
Sep 04 21:53:46.678808 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.678790    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-llxl2"
Sep 04 21:53:46.682758 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.682655    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c768d1b_c85f_459b_b512_048becacaf06.slice/crio-6c3be7beeb676244289872daa6b49bf119167cc444e692afa549cfd7b62a8718 WatchSource:0}: Error finding container 6c3be7beeb676244289872daa6b49bf119167cc444e692afa549cfd7b62a8718: Status 404 returned error can't find the container with id 6c3be7beeb676244289872daa6b49bf119167cc444e692afa549cfd7b62a8718
Sep 04 21:53:46.689588 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:53:46.689568    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5376eed9_b670_4a76_8ee4_bca13d83d09c.slice/crio-78e849072924f5951c681eefa4370540c3724a2ba384bdf8d5bfbfcfee9e0b8b WatchSource:0}: Error finding container 78e849072924f5951c681eefa4370540c3724a2ba384bdf8d5bfbfcfee9e0b8b: Status 404 returned error can't find the container with id 78e849072924f5951c681eefa4370540c3724a2ba384bdf8d5bfbfcfee9e0b8b
Sep 04 21:53:46.990633 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.989738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:46.990633 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:46.989790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:46.990633 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.989937    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:46.990633 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.990005    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:47.989984853 +0000 UTC m=+3.074369685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:46.990633 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.990463    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:46.990633 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:46.990522    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:47.990505166 +0000 UTC m=+3.074889996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:47.090726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.090685    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:47.090910 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:47.090880    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:47.090910 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:47.090900    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:47.091018 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:47.090913    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:47.091018 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:47.090976    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:48.090953646 +0000 UTC m=+3.175338485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:47.185895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.185861    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:47.347715 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.347521    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 21:48:46 +0000 UTC" deadline="2028-06-15 07:43:57.0131728 +0000 UTC"
Sep 04 21:53:47.347715 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.347561    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15585h50m9.665617858s"
Sep 04 21:53:47.397873 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.397840    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:47.398102 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:47.398074    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:47.417157 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.417082    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pj2xr" event={"ID":"266ddbfd-33be-4063-8b22-694081563177","Type":"ContainerStarted","Data":"0e6febf8bad0c8b2561abab8f1ad591af2048b8f029fbc266a78412ef4c50abb"}
Sep 04 21:53:47.420021 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.419944    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc" event={"ID":"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e","Type":"ContainerStarted","Data":"9d15180352a5fea190a951d7182205cbcd54686a2099baad75b28c9f37131374"}
Sep 04 21:53:47.442485 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.442298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nhv96" event={"ID":"36f2571c-c2b3-4884-bb88-c5182fdaffc5","Type":"ContainerStarted","Data":"776a0127dd5c7d0f64e1e0623a9c66607e86c74a6ff061c3cba674114da4d001"}
Sep 04 21:53:47.446783 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.446750    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cz75q" event={"ID":"7c768d1b-c85f-459b-b512-048becacaf06","Type":"ContainerStarted","Data":"6c3be7beeb676244289872daa6b49bf119167cc444e692afa549cfd7b62a8718"}
Sep 04 21:53:47.457919 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.457889    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pmzjn" event={"ID":"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6","Type":"ContainerStarted","Data":"9bf6c08a7ed1b6743ff14f4ee3836cc7bef0f6b9afcd0078959ed2af42d51c3c"}
Sep 04 21:53:47.460909 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.460883    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pfzcr" event={"ID":"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5","Type":"ContainerStarted","Data":"bb67904b3b8f319de84fcdd79e009538072f889e3cd456711cf8c92df32067c4"}
Sep 04 21:53:47.469135 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.469105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerStarted","Data":"69f237f785b07ee1353a44b3b040afc37b5146acf418e299b1d53224b80e9af9"}
Sep 04 21:53:47.474127 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.474097    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-llxl2" event={"ID":"5376eed9-b670-4a76-8ee4-bca13d83d09c","Type":"ContainerStarted","Data":"78e849072924f5951c681eefa4370540c3724a2ba384bdf8d5bfbfcfee9e0b8b"}
Sep 04 21:53:47.477406 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.477381    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"e0cadfd750e63b53388b834c8a6e89b9fbcb215609a3f02f765d0daa72d4491d"}
Sep 04 21:53:47.921372 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:47.921337    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:48.003302 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.003172    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:48.003302 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.003230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:48.003571 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.003425    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:48.003571 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.003493    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:50.003473386 +0000 UTC m=+5.087858279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:48.004168 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.003905    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:48.004168 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.003965    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:50.003948173 +0000 UTC m=+5.088333013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:48.103836 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.103798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:48.104080 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.104060    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:48.104162 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.104093    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:48.104162 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.104132    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:48.104273 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.104220    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:50.104199369 +0000 UTC m=+5.188584213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:48.348218 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.348105    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 21:48:46 +0000 UTC" deadline="2028-04-07 02:34:56.481028684 +0000 UTC"
Sep 04 21:53:48.348218 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.348164    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13924h41m8.132869065s"
Sep 04 21:53:48.384791 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.384758    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:48.384968 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.384912    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:48.385492 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:48.385470    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:48.385586 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:48.385566    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:49.385550 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:49.384714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:49.385550 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:49.384860    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:50.019698 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:50.019659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:50.019924 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:50.019709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:50.019924 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.019855    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:50.019924 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.019906    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:54.019893243 +0000 UTC m=+9.104278067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:50.020341 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.020237    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:50.020341 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.020284    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:54.020272208 +0000 UTC m=+9.104657032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:50.121106 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:50.120415    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:50.121106 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.120648    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:50.121106 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.120666    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:50.121106 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.120689    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:50.121106 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.120749    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:54.120730547 +0000 UTC m=+9.205115377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:50.385309 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:50.384767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:50.385309 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:50.384796    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:50.385309 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.384917    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:50.385309 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:50.385058    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:51.386399 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:51.385402    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:51.386399 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:51.385544    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:52.384616 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:52.384572    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:52.384802 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:52.384748    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:52.385195 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:52.385176    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:52.385309 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:52.385263    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:53.384699 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:53.384660    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:53.385185 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:53.384822    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:54.055362 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:54.055320    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:54.055552 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:54.055409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:54.055635 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.055575    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:54.055698 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.055647    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:02.055624917 +0000 UTC m=+17.140009763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:54.056129 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.056103    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:54.056257 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.056171    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:02.056151916 +0000 UTC m=+17.140536743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:54.157303 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:54.156652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:54.157303 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.156827    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:54.157303 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.156852    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:54.157303 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.156865    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:54.157303 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.156932    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:02.15691113 +0000 UTC m=+17.241295970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:54.384139 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:54.384061    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:54.384304 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:54.384176    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:54.384304 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.384216    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:54.384304 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:54.384260    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:55.385129 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:55.385096    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:55.385592 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:55.385177    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:56.383883 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:56.383853    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:56.383883 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:56.383878    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:56.384016 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:56.383981    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:56.384146 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:56.384118    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:57.384593 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:57.384555    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:57.385119 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:57.384694    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:53:58.384144 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:58.384111    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:53:58.384144 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:58.384139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:53:58.384414 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:58.384237    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:53:58.384414 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:58.384324    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:53:59.387650 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:53:59.387622    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:53:59.388411 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:53:59.387746    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:00.384014 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:00.383982    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:00.384197 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:00.384019    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:00.384197 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:00.384102    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:00.384302 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:00.384233    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:01.387615 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:01.387585    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:01.388001 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:01.387691    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:02.122407 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:02.122370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:02.122407 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:02.122413    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:02.122656 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.122538    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:02.122656 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.122540    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:02.122656 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.122604    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:18.122586759 +0000 UTC m=+33.206971601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:02.122656 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.122619    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:18.122612562 +0000 UTC m=+33.206997384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:02.223397 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:02.223347    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:02.223594 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.223473    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:54:02.223594 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.223500    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:54:02.223594 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.223510    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:02.223594 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.223569    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:18.223553859 +0000 UTC m=+33.307938682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:02.384315 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:02.384235    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:02.384315 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:02.384281    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:02.384578 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.384381    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:02.384578 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:02.384478    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:03.384236 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:03.384190    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:03.384679 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:03.384315    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:04.384688 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:04.384648    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:04.385127 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:04.384621    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:04.385127 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:04.384776    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:04.385127 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:04.384904    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:05.384428 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:05.384399    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:05.384618 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:05.384502    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:06.384027 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.383848    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:06.384530 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:06.384119    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:06.384530 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.383981    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:06.384530 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:06.384187    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:06.537622 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.533050    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cz75q" event={"ID":"7c768d1b-c85f-459b-b512-048becacaf06","Type":"ContainerStarted","Data":"d7a0bfa17a2cddb320578074a2bdf693ede3387d187d6d900ada9e8a0f48f884"}
Sep 04 21:54:06.537622 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.536974    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pfzcr" event={"ID":"7ddff86b-b44f-4dcc-bdbb-a6c56ac06da5","Type":"ContainerStarted","Data":"8f0e122a670ff22630fc6052db0b1ae6e843a48befee00b4cb665064e840acbf"}
Sep 04 21:54:06.540790 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.540175    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"58c56374f91b0b8ee4b418afb0d94e45558fcd6a6aef24c5801ec77f4ed77366"}
Sep 04 21:54:06.540790 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.540256    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"265eab7e3b84111870b0af08cc33759779c96401ca3eb2894f048f54d3d58380"}
Sep 04 21:54:06.552203 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.552171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal" event={"ID":"85ce9a2a03055b1e4010209d2f26585d","Type":"ContainerStarted","Data":"b9a55268fe49589102eaf52a6430779fd732f51b2c6a339e7c6d38fa82651675"}
Sep 04 21:54:06.599372 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.599247    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cz75q" podStartSLOduration=1.0344827109999999 podStartE2EDuration="20.599142537s" totalImagesPullingTime="19.564659826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:06.573669547 +0000 UTC m=+21.658054392" watchObservedRunningTime="2026-09-04 21:54:06.599142537 +0000 UTC m=+21.683527382"
Sep 04 21:54:06.599826 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.599784    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-119.ec2.internal" podStartSLOduration=20.599764727 podStartE2EDuration="20.599764727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:06.598263605 +0000 UTC m=+21.682648448" watchObservedRunningTime="2026-09-04 21:54:06.599764727 +0000 UTC m=+21.684149572"
Sep 04 21:54:06.629741 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:06.628865    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pfzcr" podStartSLOduration=1.000858561 podStartE2EDuration="20.628845085s" totalImagesPullingTime="19.627986524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:06.628230343 +0000 UTC m=+21.712615196" watchObservedRunningTime="2026-09-04 21:54:06.628845085 +0000 UTC m=+21.713229930"
Sep 04 21:54:07.387920 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.387891    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:07.388640 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:07.388010    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:07.555513 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.555426    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pmzjn" event={"ID":"0b8f1fce-1378-4ae4-9b93-bbde1a048fa6","Type":"ContainerStarted","Data":"ddf99c3dfd6dba5adf9a77ea6c19f6d6b041b14869cb01046c03f3d9e3eee1b1"}
Sep 04 21:54:07.556890 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.556868    2565 generic.go:356] "Generic (PLEG): container finished" podID="dad701d3-4293-4879-9296-70fcfc8100fd" containerID="2c159153b5678741ac01994127a635cb67a544f8a603f5a5dde6a04ff0109a48" exitCode=0
Sep 04 21:54:07.557022 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.556944    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerDied","Data":"2c159153b5678741ac01994127a635cb67a544f8a603f5a5dde6a04ff0109a48"}
Sep 04 21:54:07.558437 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.558407    2565 generic.go:356] "Generic (PLEG): container finished" podID="e4eb6a818c7d4b42d870eb4178268581" containerID="1fa98f18aeb8d21e016d12519c48bbdb62e866d3e7867d0d35d75ed26897fcac" exitCode=0
Sep 04 21:54:07.558531 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.558459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal" event={"ID":"e4eb6a818c7d4b42d870eb4178268581","Type":"ContainerDied","Data":"1fa98f18aeb8d21e016d12519c48bbdb62e866d3e7867d0d35d75ed26897fcac"}
Sep 04 21:54:07.559959 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.559940    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-llxl2" event={"ID":"5376eed9-b670-4a76-8ee4-bca13d83d09c","Type":"ContainerStarted","Data":"9b84d704f391f23651e816b6361451ed140b83d099c2b8fc8a1177079ababd34"}
Sep 04 21:54:07.562930 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.562910    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"66e1274e46e2ce65688eca5dd3b3eeec9ef284926078467247306a22f5eccd4a"}
Sep 04 21:54:07.563028 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.562984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"fb8b2a949320c897e5e91f6687aa40b280e81fe0764ebcf157b0eeda95e7bb99"}
Sep 04 21:54:07.563077 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.563032    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"48a6ad588a9ba9bc190d01fa3000247071e6b0cb3eeedf31d0da3cbd86b978e0"}
Sep 04 21:54:07.563115 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.563079    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"b304162fa328416289e2f2f22e6765585ff3e0f3a6142e4592277e40e0115bec"}
Sep 04 21:54:07.564379 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.564357    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pj2xr" event={"ID":"266ddbfd-33be-4063-8b22-694081563177","Type":"ContainerStarted","Data":"ab3eed6827ea1faefac5ca1c534bb373ef84e4c23ad3b1dfbeeb6a374cdac7a1"}
Sep 04 21:54:07.565726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.565699    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc" event={"ID":"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e","Type":"ContainerStarted","Data":"f11eb077e63398a397c691130ac6bbbac5485f75f430b45d4601941ca09cb1a2"}
Sep 04 21:54:07.581982 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.581949    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pmzjn" podStartSLOduration=2.041844751 podStartE2EDuration="21.581935041s" totalImagesPullingTime="19.54009029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.581311517 +0000 UTC m=+22.665696360" watchObservedRunningTime="2026-09-04 21:54:07.581935041 +0000 UTC m=+22.666319886"
Sep 04 21:54:07.653216 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.653077    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-llxl2" podStartSLOduration=2.124815338 podStartE2EDuration="21.653058304s" totalImagesPullingTime="19.528242966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.621780392 +0000 UTC m=+22.706165235" watchObservedRunningTime="2026-09-04 21:54:07.653058304 +0000 UTC m=+22.737443151"
Sep 04 21:54:07.682873 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.682832    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pj2xr" podStartSLOduration=2.10012924 podStartE2EDuration="21.682818933s" totalImagesPullingTime="19.582689693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.682454641 +0000 UTC m=+22.766839485" watchObservedRunningTime="2026-09-04 21:54:07.682818933 +0000 UTC m=+22.767203776"
Sep 04 21:54:07.937710 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:07.937687    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 04 21:54:08.378984 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.378876    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-04T21:54:07.937705923Z","UUID":"0a8dba60-81bc-4f7c-8b3f-fb4b2227f38b","Handler":null,"Name":"","Endpoint":""}
Sep 04 21:54:08.380787 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.380761    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 04 21:54:08.380913 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.380795    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 04 21:54:08.383907 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.383882    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:08.384016 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.383892    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:08.384016 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:08.383992    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:08.384112 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:08.384048    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:08.570160 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.570123    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal" event={"ID":"e4eb6a818c7d4b42d870eb4178268581","Type":"ContainerStarted","Data":"8bbf926ee0d455a1a01734b2eb94562be2517011b3cf94641bd97ecb854b6928"}
Sep 04 21:54:08.573114 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.573083    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc" event={"ID":"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e","Type":"ContainerStarted","Data":"fd7c8d73f49da35dc222ce298e439a75b979a41eaf54b896cdbc3bef4239fb2c"}
Sep 04 21:54:08.574877 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.574852    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nhv96" event={"ID":"36f2571c-c2b3-4884-bb88-c5182fdaffc5","Type":"ContainerStarted","Data":"bc0d79db36b65723a66a27085b3380c400ca1c7ed22a41ffa1ad6c71a16082f2"}
Sep 04 21:54:08.597536 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.597488    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-119.ec2.internal" podStartSLOduration=22.597469685 podStartE2EDuration="22.597469685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:08.597233763 +0000 UTC m=+23.681618607" watchObservedRunningTime="2026-09-04 21:54:08.597469685 +0000 UTC m=+23.681854529"
Sep 04 21:54:08.814161 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.814058    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:54:08.905693 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.905662    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:54:08.906208 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.906183    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:54:08.924819 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:08.924766    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nhv96" podStartSLOduration=3.315161121 podStartE2EDuration="22.924747576s" totalImagesPullingTime="19.609586455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:08.621624685 +0000 UTC m=+23.706009530" watchObservedRunningTime="2026-09-04 21:54:08.924747576 +0000 UTC m=+24.009132423"
Sep 04 21:54:09.384018 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:09.383942    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:09.385624 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:09.385069    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:09.583522 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:09.583485    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"f293af58251d0d94070e8d300b7934ba603a7b170d983fd297678423f848ecad"}
Sep 04 21:54:09.586150 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:09.585951    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc" event={"ID":"2ecbaa34-70f9-4014-9d8a-f6392fea9f8e","Type":"ContainerStarted","Data":"b6a3a5cee0d23cdff34ebe32a82a9f07e41497aee5aa274b757f707ec78d0f6d"}
Sep 04 21:54:09.587202 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:09.587180    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pmzjn"
Sep 04 21:54:09.603960 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:09.603919    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f6nc" podStartSLOduration=1.6814471850000001 podStartE2EDuration="23.603906738s" totalImagesPullingTime="21.922459553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:09.603404474 +0000 UTC m=+24.687789319" watchObservedRunningTime="2026-09-04 21:54:09.603906738 +0000 UTC m=+24.688291581"
Sep 04 21:54:10.384588 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:10.384550    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:10.384759 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:10.384568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:10.384759 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:10.384684    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:10.384759 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:10.384723    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:10.592212 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:10.592185    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pj2xr_266ddbfd-33be-4063-8b22-694081563177/dns-node-resolver/0.log"
Sep 04 21:54:11.384083 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:11.384055    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:11.384242 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:11.384172    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:11.754005 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:11.753924    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-llxl2_5376eed9-b670-4a76-8ee4-bca13d83d09c/node-ca/0.log"
Sep 04 21:54:12.384369 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.384329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:12.384466 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.384329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:12.384554 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:12.384449    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:12.384600 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:12.384575    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:12.595711 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.595469    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerStarted","Data":"8b470d45e2601410bfc076fb24524bc0f16042e436aef76b8326279461b2b5b7"}
Sep 04 21:54:12.600141 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.600112    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" event={"ID":"73a8e6b9-c75a-4cf9-8a26-5fa6c3ba9a71","Type":"ContainerStarted","Data":"d00649a2a9a92f278d77b18a092141fbb157a6e2121262088aac07b713fb4ce2"}
Sep 04 21:54:12.603155 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.600914    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:54:12.603155 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.601131    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:54:12.603155 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.601230    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:54:12.621124 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.621101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:54:12.621254 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.621241    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:54:12.654221 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:12.654178    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb" podStartSLOduration=6.993666199 podStartE2EDuration="26.654165344s" totalImagesPullingTime="19.660499145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:12.653923197 +0000 UTC m=+27.738308040" watchObservedRunningTime="2026-09-04 21:54:12.654165344 +0000 UTC m=+27.738550184"
Sep 04 21:54:13.384726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:13.384690    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:13.385085 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:13.384791    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:13.604734 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:13.604688    2565 generic.go:356] "Generic (PLEG): container finished" podID="dad701d3-4293-4879-9296-70fcfc8100fd" containerID="8b470d45e2601410bfc076fb24524bc0f16042e436aef76b8326279461b2b5b7" exitCode=0
Sep 04 21:54:13.604890 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:13.604763    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerDied","Data":"8b470d45e2601410bfc076fb24524bc0f16042e436aef76b8326279461b2b5b7"}
Sep 04 21:54:14.383905 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.383883    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:14.383966 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.383916    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:14.384018 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:14.384001    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:14.384138 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:14.384120    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:14.456212 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.456184    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8nmrf"]
Sep 04 21:54:14.461839 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.461776    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f96gh"]
Sep 04 21:54:14.461967 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.461889    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:14.462028 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:14.461987    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:14.462428 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.462408    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x6rqt"]
Sep 04 21:54:14.608508 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.608479    2565 generic.go:356] "Generic (PLEG): container finished" podID="dad701d3-4293-4879-9296-70fcfc8100fd" containerID="71b4b55596c1e7a8de7e2ecd4a4c3085853b2b2444e116646a1c3769e3397522" exitCode=0
Sep 04 21:54:14.608705 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.608572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerDied","Data":"71b4b55596c1e7a8de7e2ecd4a4c3085853b2b2444e116646a1c3769e3397522"}
Sep 04 21:54:14.608768 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.608707    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:14.608816 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:14.608798    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:14.609292 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:14.609264    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:14.610065 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:14.610039    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:15.613242 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:15.613210    2565 generic.go:356] "Generic (PLEG): container finished" podID="dad701d3-4293-4879-9296-70fcfc8100fd" containerID="60b717ea5ce56fe228908359097ab678d05924d7091c32382016e759bd14062e" exitCode=0
Sep 04 21:54:15.613606 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:15.613258    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerDied","Data":"60b717ea5ce56fe228908359097ab678d05924d7091c32382016e759bd14062e"}
Sep 04 21:54:16.384111 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:16.384075    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:16.384243 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:16.384124    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:16.384243 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:16.384143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:16.384353 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:16.384269    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:16.384816 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:16.384415    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:16.387197 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:16.384972    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:18.132845 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:18.132813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:18.132845 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:18.132849    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:18.133269 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.132933    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:18.133269 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.132980    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:18.133269 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.133001    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret podName:c6b2f1f2-f011-4d5e-b80c-f15c6add8565 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:50.132968901 +0000 UTC m=+65.217353723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret") pod "global-pull-secret-syncer-f96gh" (UID: "c6b2f1f2-f011-4d5e-b80c-f15c6add8565") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:18.133269 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.133044    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs podName:453903df-4411-410d-a1b5-9a513da936c4 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:50.133024992 +0000 UTC m=+65.217409828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs") pod "network-metrics-daemon-8nmrf" (UID: "453903df-4411-410d-a1b5-9a513da936c4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:18.234000 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:18.233968    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:18.234157 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.234101    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:54:18.234157 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.234123    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:54:18.234157 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.234135    2565 projected.go:196] Error preparing data for projected volume kube-api-access-bhcbz for pod openshift-network-diagnostics/network-check-target-x6rqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:18.234318 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.234199    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz podName:977ad0ad-b542-43b5-b623-21e789286a10 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:50.234178486 +0000 UTC m=+65.318563325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhcbz" (UniqueName: "kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz") pod "network-check-target-x6rqt" (UID: "977ad0ad-b542-43b5-b623-21e789286a10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:18.384647 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:18.384608    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:18.384797 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:18.384607    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:18.384797 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.384749    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:18.384797 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:18.384607    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:18.384937 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.384819    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:18.384937 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:18.384894    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:20.384634 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:20.384597    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:20.384634 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:20.384620    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:20.385065 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:20.384695    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:20.385065 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:20.384752    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:20.385065 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:20.384790    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:20.385065 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:20.384854    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:20.627142 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:20.627104    2565 generic.go:356] "Generic (PLEG): container finished" podID="dad701d3-4293-4879-9296-70fcfc8100fd" containerID="94a195e7eaf505d27fa415b344cd3f8b140c8d5c2faa7b533307674f1a627865" exitCode=0
Sep 04 21:54:20.627296 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:20.627163    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerDied","Data":"94a195e7eaf505d27fa415b344cd3f8b140c8d5c2faa7b533307674f1a627865"}
Sep 04 21:54:21.632656 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:21.632626    2565 generic.go:356] "Generic (PLEG): container finished" podID="dad701d3-4293-4879-9296-70fcfc8100fd" containerID="fba10f8062cca6fe9fc0ec2b8d3dcbbae77bb028f5ee05791bb92609daa49d56" exitCode=0
Sep 04 21:54:21.632991 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:21.632667    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerDied","Data":"fba10f8062cca6fe9fc0ec2b8d3dcbbae77bb028f5ee05791bb92609daa49d56"}
Sep 04 21:54:22.384617 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:22.384440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:22.384838 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:22.384440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:22.384838 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:22.384698    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:22.384838 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:22.384440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:22.384838 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:22.384792    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:22.385003 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:22.384862    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:22.638524 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:22.638446    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" event={"ID":"dad701d3-4293-4879-9296-70fcfc8100fd","Type":"ContainerStarted","Data":"22d05ecbe67594681cb7b23bfc31eac7ab6b467ac69ba1d00943da84274a8d69"}
Sep 04 21:54:22.668230 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:22.667075    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vrw2k" podStartSLOduration=6.586342567 podStartE2EDuration="36.667054053s" totalImagesPullingTime="30.080711486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:22.665927295 +0000 UTC m=+37.750312140" watchObservedRunningTime="2026-09-04 21:54:22.667054053 +0000 UTC m=+37.751438897"
Sep 04 21:54:24.384501 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:24.384469    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:24.384501 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:24.384495    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:24.384895 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:24.384575    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:24.384895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:24.384640    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:24.384895 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:24.384762    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:24.384895 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:24.384872    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:26.384808 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:26.384772    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:26.385158 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:26.384789    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:26.385158 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:26.384894    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:26.385158 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:26.384951    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:26.385158 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:26.385023    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:26.385158 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:26.385115    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:28.384466 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:28.384441    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:28.384466 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:28.384448    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:28.384466 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:28.384441    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:28.384892 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:28.384566    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:28.384892 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:28.384606    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:28.384892 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:28.384680    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:30.384138 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:30.384108    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:30.384540 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:30.384113    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:30.384540 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:30.384205    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:30.384540 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:30.384229    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:30.384540 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:30.384298    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:30.384540 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:30.384382    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:32.384487 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:32.384460    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:32.384836 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:32.384462    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:32.384836 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:32.384548    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:32.384836 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:32.384462    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:32.384836 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:32.384634    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:32.384836 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:32.384721    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:34.384651 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:34.384617    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:34.385108 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:34.384706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:34.385108 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:34.384723    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:34.385108 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:34.384710    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x6rqt" podUID="977ad0ad-b542-43b5-b623-21e789286a10"
Sep 04 21:54:34.385108 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:34.384826    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nmrf" podUID="453903df-4411-410d-a1b5-9a513da936c4"
Sep 04 21:54:34.385108 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:34.384893    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f96gh" podUID="c6b2f1f2-f011-4d5e-b80c-f15c6add8565"
Sep 04 21:54:36.186285 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.186255    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-119.ec2.internal" event="NodeReady"
Sep 04 21:54:36.186679 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.186376    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 04 21:54:36.237814 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.237786    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-89dc6"]
Sep 04 21:54:36.259471 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.259448    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-glw6j"]
Sep 04 21:54:36.259607 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.259580    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.262551 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.262526    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 04 21:54:36.262658 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.262537    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5hzgq\""
Sep 04 21:54:36.262709 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.262676    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 04 21:54:36.266740 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.266724    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a03178a-a0f7-4721-9096-725914c8ebd2-metrics-tls\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.266810 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.266770    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a03178a-a0f7-4721-9096-725914c8ebd2-tmp-dir\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.266863 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.266844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhfc9\" (UniqueName: \"kubernetes.io/projected/3a03178a-a0f7-4721-9096-725914c8ebd2-kube-api-access-nhfc9\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.266921 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.266876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a03178a-a0f7-4721-9096-725914c8ebd2-config-volume\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.278710 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.278689    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89dc6"]
Sep 04 21:54:36.278796 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.278719    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-glw6j"]
Sep 04 21:54:36.278796 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.278735    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-477ql"]
Sep 04 21:54:36.278865 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.278797    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.281695 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.281675    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 04 21:54:36.281806 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.281791    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dmz5v\""
Sep 04 21:54:36.281911 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.281895    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 04 21:54:36.282129 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.282116    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 04 21:54:36.300858 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.300831    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-477ql"]
Sep 04 21:54:36.300948 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.300932    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.303032 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.302983    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 04 21:54:36.303291 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.303273    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 04 21:54:36.303291 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.303287    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 04 21:54:36.303505 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.303488    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9gp7g\""
Sep 04 21:54:36.303578 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.303520    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 04 21:54:36.367150 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367123    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a03178a-a0f7-4721-9096-725914c8ebd2-metrics-tls\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.367272 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367155    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.367272 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367190    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp9wx\" (UniqueName: \"kubernetes.io/projected/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-kube-api-access-bp9wx\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.367272 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a03178a-a0f7-4721-9096-725914c8ebd2-tmp-dir\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.367398 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-data-volume\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.367444 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-crio-socket\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.367478 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367455    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.367515 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhfc9\" (UniqueName: \"kubernetes.io/projected/3a03178a-a0f7-4721-9096-725914c8ebd2-kube-api-access-nhfc9\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.367515 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367505    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhdr\" (UniqueName: \"kubernetes.io/projected/3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194-kube-api-access-6rhdr\") pod \"ingress-canary-glw6j\" (UID: \"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194\") " pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.367589 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a03178a-a0f7-4721-9096-725914c8ebd2-config-volume\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.367638 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194-cert\") pod \"ingress-canary-glw6j\" (UID: \"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194\") " pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.367638 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367600    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a03178a-a0f7-4721-9096-725914c8ebd2-tmp-dir\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.368012 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.367991    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a03178a-a0f7-4721-9096-725914c8ebd2-config-volume\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.371083 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.371064    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a03178a-a0f7-4721-9096-725914c8ebd2-metrics-tls\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.375554 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.375530    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhfc9\" (UniqueName: \"kubernetes.io/projected/3a03178a-a0f7-4721-9096-725914c8ebd2-kube-api-access-nhfc9\") pod \"dns-default-89dc6\" (UID: \"3a03178a-a0f7-4721-9096-725914c8ebd2\") " pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.384492 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.384478    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:36.384566 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.384492    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:36.384566 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.384478    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:36.387161 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.387126    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-268hq\""
Sep 04 21:54:36.387297 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.387271    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xzsq2\""
Sep 04 21:54:36.387416 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.387303    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 21:54:36.387416 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.387331    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 21:54:36.387416 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.387305    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 21:54:36.387557 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.387457    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 21:54:36.468758 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.468706    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.468758 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.468745    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhdr\" (UniqueName: \"kubernetes.io/projected/3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194-kube-api-access-6rhdr\") pod \"ingress-canary-glw6j\" (UID: \"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194\") " pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.468888 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.468769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194-cert\") pod \"ingress-canary-glw6j\" (UID: \"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194\") " pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.468888 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.468830    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.468996 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.468975    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp9wx\" (UniqueName: \"kubernetes.io/projected/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-kube-api-access-bp9wx\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.469050 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.469030    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-data-volume\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.469109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.469063    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-crio-socket\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.469308 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.469287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-crio-socket\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.469494 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.469472    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-data-volume\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.469591 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.469543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.471093 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.471065    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.471290 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.471275    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194-cert\") pod \"ingress-canary-glw6j\" (UID: \"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194\") " pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.483908 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.483885    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp9wx\" (UniqueName: \"kubernetes.io/projected/a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b-kube-api-access-bp9wx\") pod \"insights-runtime-extractor-477ql\" (UID: \"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b\") " pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.484715 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.484695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhdr\" (UniqueName: \"kubernetes.io/projected/3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194-kube-api-access-6rhdr\") pod \"ingress-canary-glw6j\" (UID: \"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194\") " pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.568392 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.568365    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:36.585745 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.585726    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-glw6j"
Sep 04 21:54:36.609194 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.609168    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-477ql"
Sep 04 21:54:36.756991 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.756918    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"]
Sep 04 21:54:36.758823 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.758802    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:36.760872 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.760853    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 04 21:54:36.760977 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.760917    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-d9gzs\""
Sep 04 21:54:36.767728 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.767707    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"]
Sep 04 21:54:36.771258 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.771238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-457w5\" (UID: \"95f7c41a-75b6-403a-91cc-73ce1dafc5e6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:36.853264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.853236    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89dc6"]
Sep 04 21:54:36.857231 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:36.857204    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a03178a_a0f7_4721_9096_725914c8ebd2.slice/crio-969e66e0d2ec921c9c5d3f9d1e7a7cd0ec7fb891bd33f6c5be412719ba171476 WatchSource:0}: Error finding container 969e66e0d2ec921c9c5d3f9d1e7a7cd0ec7fb891bd33f6c5be412719ba171476: Status 404 returned error can't find the container with id 969e66e0d2ec921c9c5d3f9d1e7a7cd0ec7fb891bd33f6c5be412719ba171476
Sep 04 21:54:36.872428 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.872406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-457w5\" (UID: \"95f7c41a-75b6-403a-91cc-73ce1dafc5e6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:36.872564 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:36.872546    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 21:54:36.872644 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:36.872630    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates podName:95f7c41a-75b6-403a-91cc-73ce1dafc5e6 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:37.372609019 +0000 UTC m=+52.456993858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-457w5" (UID: "95f7c41a-75b6-403a-91cc-73ce1dafc5e6") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 21:54:36.876899 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.876879    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-glw6j"]
Sep 04 21:54:36.880653 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:36.880630    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd6a430_21d9_4b54_ab7b_a4a1cb9d6194.slice/crio-391c894f390cebf75d5935cf97fd053e97f62b39384458d0c5052cbd4a6c1a82 WatchSource:0}: Error finding container 391c894f390cebf75d5935cf97fd053e97f62b39384458d0c5052cbd4a6c1a82: Status 404 returned error can't find the container with id 391c894f390cebf75d5935cf97fd053e97f62b39384458d0c5052cbd4a6c1a82
Sep 04 21:54:36.897567 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:36.897525    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-477ql"]
Sep 04 21:54:36.901498 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:36.901477    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64ac37b_972f_4bd5_bdbc_be5f5a6ad89b.slice/crio-a3881bf98d98b994416a8bde09bb9e4effe2ebd6383bcf048102353dac37abc5 WatchSource:0}: Error finding container a3881bf98d98b994416a8bde09bb9e4effe2ebd6383bcf048102353dac37abc5: Status 404 returned error can't find the container with id a3881bf98d98b994416a8bde09bb9e4effe2ebd6383bcf048102353dac37abc5
Sep 04 21:54:37.376498 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:37.376456    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-457w5\" (UID: \"95f7c41a-75b6-403a-91cc-73ce1dafc5e6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:37.376877 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:37.376609    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 21:54:37.376877 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:37.376675    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates podName:95f7c41a-75b6-403a-91cc-73ce1dafc5e6 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:38.376659848 +0000 UTC m=+53.461044669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-457w5" (UID: "95f7c41a-75b6-403a-91cc-73ce1dafc5e6") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 21:54:37.678138 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:37.678098    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89dc6" event={"ID":"3a03178a-a0f7-4721-9096-725914c8ebd2","Type":"ContainerStarted","Data":"969e66e0d2ec921c9c5d3f9d1e7a7cd0ec7fb891bd33f6c5be412719ba171476"}
Sep 04 21:54:37.680109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:37.679796    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-477ql" event={"ID":"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b","Type":"ContainerStarted","Data":"4a02c2e86fe9fd8f5f843839e5eba53bf18bc3a903cb06ccef9172f7e732bf21"}
Sep 04 21:54:37.680109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:37.679899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-477ql" event={"ID":"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b","Type":"ContainerStarted","Data":"a3881bf98d98b994416a8bde09bb9e4effe2ebd6383bcf048102353dac37abc5"}
Sep 04 21:54:37.681666 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:37.681640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-glw6j" event={"ID":"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194","Type":"ContainerStarted","Data":"391c894f390cebf75d5935cf97fd053e97f62b39384458d0c5052cbd4a6c1a82"}
Sep 04 21:54:38.387452 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:38.387300    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-457w5\" (UID: \"95f7c41a-75b6-403a-91cc-73ce1dafc5e6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:38.390295 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:38.390262    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/95f7c41a-75b6-403a-91cc-73ce1dafc5e6-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-457w5\" (UID: \"95f7c41a-75b6-403a-91cc-73ce1dafc5e6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:38.567937 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:38.567891    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:38.686848 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:38.686752    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-477ql" event={"ID":"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b","Type":"ContainerStarted","Data":"526bd3626dc2b6403fe9281b9dd9dc1144da2c12352ce06323d06f34e160ef61"}
Sep 04 21:54:39.499468 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.499396    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"]
Sep 04 21:54:39.691498 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.691433    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-glw6j" event={"ID":"3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194","Type":"ContainerStarted","Data":"e41b529d41b4ef602b91673cc23b326f5033ba53e7b2a2f01371c0f1939d00ba"}
Sep 04 21:54:39.693512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.693482    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89dc6" event={"ID":"3a03178a-a0f7-4721-9096-725914c8ebd2","Type":"ContainerStarted","Data":"1879bf87292a8eba96a140d1d94e26c2ab06b7b82fd9300b8e51a02d2e7d9f86"}
Sep 04 21:54:39.693651 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.693532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89dc6" event={"ID":"3a03178a-a0f7-4721-9096-725914c8ebd2","Type":"ContainerStarted","Data":"bbfdb18de2a699b5fde2320d43a17c80d94ba779452c3e9c25b2e46b97481cbb"}
Sep 04 21:54:39.693651 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.693558    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:39.694770 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.694744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5" event={"ID":"95f7c41a-75b6-403a-91cc-73ce1dafc5e6","Type":"ContainerStarted","Data":"782aa8def40f283b19b1f97ff14706b07d7d56d7541fc7eccfc496118daca593"}
Sep 04 21:54:39.707915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.707877    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-glw6j" podStartSLOduration=1.355006571 podStartE2EDuration="3.707863512s" totalImagesPullingTime="2.352856941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:39.707284373 +0000 UTC m=+54.791669218" watchObservedRunningTime="2026-09-04 21:54:39.707863512 +0000 UTC m=+54.792248355"
Sep 04 21:54:39.726128 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:39.726079    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-89dc6" podStartSLOduration=1.35927029 podStartE2EDuration="3.726062576s" totalImagesPullingTime="2.366792286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:39.724581279 +0000 UTC m=+54.808966123" watchObservedRunningTime="2026-09-04 21:54:39.726062576 +0000 UTC m=+54.810447421"
Sep 04 21:54:40.700890 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:40.700851    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-477ql" event={"ID":"a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b","Type":"ContainerStarted","Data":"53e0e1114d2b3fff1893282cdedc22c58ca1ebcb3fae6e73447ca28617d31f53"}
Sep 04 21:54:40.723374 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:40.723061    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-477ql" podStartSLOduration=1.626956593 podStartE2EDuration="4.723039808s" totalImagesPullingTime="3.096083215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:54:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:40.721991314 +0000 UTC m=+55.806376158" watchObservedRunningTime="2026-09-04 21:54:40.723039808 +0000 UTC m=+55.807424653"
Sep 04 21:54:41.705373 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:41.705317    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5" event={"ID":"95f7c41a-75b6-403a-91cc-73ce1dafc5e6","Type":"ContainerStarted","Data":"83677d6c22d0339af761218b0ae8eb1a90091afe455d58f468ac9d6f67fc662f"}
Sep 04 21:54:41.725822 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:41.725779    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5" podStartSLOduration=4.3411622770000005 podStartE2EDuration="5.725763406s" totalImagesPullingTime="1.384601129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:41.725002293 +0000 UTC m=+56.809387136" watchObservedRunningTime="2026-09-04 21:54:41.725763406 +0000 UTC m=+56.810148250"
Sep 04 21:54:42.707975 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:42.707940    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:42.712303 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:42.712280    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-457w5"
Sep 04 21:54:43.824112 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.824074    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"]
Sep 04 21:54:43.826843 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.826822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:43.830249 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.830226    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 04 21:54:43.830564 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.830545    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 04 21:54:43.830658 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.830574    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tkzhk\""
Sep 04 21:54:43.830658 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.830588    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 04 21:54:43.830834 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.830820    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 04 21:54:43.830912 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.830894    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 04 21:54:43.839125 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.839106    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"]
Sep 04 21:54:43.924641 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.924612    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8ww9\" (UniqueName: \"kubernetes.io/projected/85d075e4-6427-45bf-9553-65175191f48b-kube-api-access-h8ww9\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:43.924800 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.924649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:43.924800 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.924669    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85d075e4-6427-45bf-9553-65175191f48b-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:43.924800 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:43.924770    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.025921 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.025888    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8ww9\" (UniqueName: \"kubernetes.io/projected/85d075e4-6427-45bf-9553-65175191f48b-kube-api-access-h8ww9\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.025985 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.025930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.025985 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.025958    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85d075e4-6427-45bf-9553-65175191f48b-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.026048 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.026019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.026081 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:44.026041    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 04 21:54:44.026109 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:44.026095    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-tls podName:85d075e4-6427-45bf-9553-65175191f48b nodeName:}" failed. No retries permitted until 2026-09-04 21:54:44.526080017 +0000 UTC m=+59.610464839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-5qkbg" (UID: "85d075e4-6427-45bf-9553-65175191f48b") : secret "prometheus-operator-tls" not found
Sep 04 21:54:44.026639 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.026621    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85d075e4-6427-45bf-9553-65175191f48b-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.028536 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.028518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.036307 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.036288    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8ww9\" (UniqueName: \"kubernetes.io/projected/85d075e4-6427-45bf-9553-65175191f48b-kube-api-access-h8ww9\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.529883 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.529819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.532290 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.532268    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85d075e4-6427-45bf-9553-65175191f48b-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5qkbg\" (UID: \"85d075e4-6427-45bf-9553-65175191f48b\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:44.619205 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.619182    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cffb"
Sep 04 21:54:44.735042 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:44.735014    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"
Sep 04 21:54:45.000567 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:45.000393    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5qkbg"]
Sep 04 21:54:45.002914 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:45.002885    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d075e4_6427_45bf_9553_65175191f48b.slice/crio-191c50d98e54c7131a5215111d6fdb522ed53f3f9fdccfcd4b73dc0bb058400e WatchSource:0}: Error finding container 191c50d98e54c7131a5215111d6fdb522ed53f3f9fdccfcd4b73dc0bb058400e: Status 404 returned error can't find the container with id 191c50d98e54c7131a5215111d6fdb522ed53f3f9fdccfcd4b73dc0bb058400e
Sep 04 21:54:45.717264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:45.717224    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg" event={"ID":"85d075e4-6427-45bf-9553-65175191f48b","Type":"ContainerStarted","Data":"191c50d98e54c7131a5215111d6fdb522ed53f3f9fdccfcd4b73dc0bb058400e"}
Sep 04 21:54:46.724328 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:46.724246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg" event={"ID":"85d075e4-6427-45bf-9553-65175191f48b","Type":"ContainerStarted","Data":"6dabe2e50f22b1835d18166080ef74ae142ad550031ffa72ca6f270b12147460"}
Sep 04 21:54:46.724757 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:46.724363    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg" event={"ID":"85d075e4-6427-45bf-9553-65175191f48b","Type":"ContainerStarted","Data":"e21bce628a5b70fa1de3c51f11358bc3dfcf5692394fd611f1066c0d616ccab1"}
Sep 04 21:54:46.741461 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:46.741417    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-5qkbg" podStartSLOduration=2.406307269 podStartE2EDuration="3.741402457s" totalImagesPullingTime="1.335095188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:46.740896422 +0000 UTC m=+61.825281267" watchObservedRunningTime="2026-09-04 21:54:46.741402457 +0000 UTC m=+61.825787326"
Sep 04 21:54:48.254517 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.254478    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k6qht"]
Sep 04 21:54:48.256645 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.256624    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.258800 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.258766    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 04 21:54:48.259085 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.259066    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 04 21:54:48.259165 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.259098    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 04 21:54:48.259226 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.259171    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2cnfl\""
Sep 04 21:54:48.353915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.353886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-accelerators-collector-config\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354073 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.353924    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-textfile\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354073 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.353995    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-tls\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354073 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.354039    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-sys\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354073 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.354057    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbghs\" (UniqueName: \"kubernetes.io/projected/29723d57-fa71-4252-8517-89db77537ec3-kube-api-access-lbghs\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354257 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.354078    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-root\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354257 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.354146    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-wtmp\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354257 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.354209    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29723d57-fa71-4252-8517-89db77537ec3-metrics-client-ca\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.354257 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.354225    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455459    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-wtmp\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455491 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29723d57-fa71-4252-8517-89db77537ec3-metrics-client-ca\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455709 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455709 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455575    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-accelerators-collector-config\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455709 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-textfile\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455709 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-tls\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455709 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455652    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-wtmp\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455712    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-sys\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:48.455734    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbghs\" (UniqueName: \"kubernetes.io/projected/29723d57-fa71-4252-8517-89db77537ec3-kube-api-access-lbghs\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-root\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: E0904 21:54:48.455795    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-tls podName:29723d57-fa71-4252-8517-89db77537ec3 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:48.955778215 +0000 UTC m=+64.040163052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-tls") pod "node-exporter-k6qht" (UID: "29723d57-fa71-4252-8517-89db77537ec3") : secret "node-exporter-tls" not found
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455806    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-root\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.455956 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29723d57-fa71-4252-8517-89db77537ec3-sys\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.456310 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.455982    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-textfile\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.456310 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.456127    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29723d57-fa71-4252-8517-89db77537ec3-metrics-client-ca\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.456444 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.456387    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-accelerators-collector-config\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.458055 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.458039    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.472258 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.472239    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbghs\" (UniqueName: \"kubernetes.io/projected/29723d57-fa71-4252-8517-89db77537ec3-kube-api-access-lbghs\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.960264 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.960231    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-tls\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:48.962605 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:48.962574    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/29723d57-fa71-4252-8517-89db77537ec3-node-exporter-tls\") pod \"node-exporter-k6qht\" (UID: \"29723d57-fa71-4252-8517-89db77537ec3\") " pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:49.165653 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:49.165619    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k6qht"
Sep 04 21:54:49.198103 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:49.198065    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29723d57_fa71_4252_8517_89db77537ec3.slice/crio-7a5c6828c352db4ef0cc243c8f8d9424d97d64482b3e83a3845adbc5095d2872 WatchSource:0}: Error finding container 7a5c6828c352db4ef0cc243c8f8d9424d97d64482b3e83a3845adbc5095d2872: Status 404 returned error can't find the container with id 7a5c6828c352db4ef0cc243c8f8d9424d97d64482b3e83a3845adbc5095d2872
Sep 04 21:54:49.703362 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:49.703301    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-89dc6"
Sep 04 21:54:49.735572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:49.735536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6qht" event={"ID":"29723d57-fa71-4252-8517-89db77537ec3","Type":"ContainerStarted","Data":"7a5c6828c352db4ef0cc243c8f8d9424d97d64482b3e83a3845adbc5095d2872"}
Sep 04 21:54:50.169652 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.169618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:50.169812 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.169660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:50.171928 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.171910    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 21:54:50.172654 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.172632    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 21:54:50.182516 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.182492    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6b2f1f2-f011-4d5e-b80c-f15c6add8565-original-pull-secret\") pod \"global-pull-secret-syncer-f96gh\" (UID: \"c6b2f1f2-f011-4d5e-b80c-f15c6add8565\") " pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:50.182601 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.182532    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453903df-4411-410d-a1b5-9a513da936c4-metrics-certs\") pod \"network-metrics-daemon-8nmrf\" (UID: \"453903df-4411-410d-a1b5-9a513da936c4\") " pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:50.194655 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.194612    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f96gh"
Sep 04 21:54:50.209865 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.209845    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-268hq\""
Sep 04 21:54:50.219815 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.219795    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nmrf"
Sep 04 21:54:50.270411 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.270385    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:50.272727 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.272693    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 21:54:50.283002 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.282785    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 21:54:50.294250 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.294220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhcbz\" (UniqueName: \"kubernetes.io/projected/977ad0ad-b542-43b5-b623-21e789286a10-kube-api-access-bhcbz\") pod \"network-check-target-x6rqt\" (UID: \"977ad0ad-b542-43b5-b623-21e789286a10\") " pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:50.484539 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.484467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f96gh"]
Sep 04 21:54:50.506388 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.506364    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xzsq2\""
Sep 04 21:54:50.514544 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.514518    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:50.524609 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.524583    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8nmrf"]
Sep 04 21:54:50.581778 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:50.581755    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b2f1f2_f011_4d5e_b80c_f15c6add8565.slice/crio-9a6549e52a5cdc92c47dd061dea76267fd70f1e8281c65c3da950fda67b9ad46 WatchSource:0}: Error finding container 9a6549e52a5cdc92c47dd061dea76267fd70f1e8281c65c3da950fda67b9ad46: Status 404 returned error can't find the container with id 9a6549e52a5cdc92c47dd061dea76267fd70f1e8281c65c3da950fda67b9ad46
Sep 04 21:54:50.582380 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:50.582358    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453903df_4411_410d_a1b5_9a513da936c4.slice/crio-d1be76123f3405b8adcc7212870ea27124f0d24cd01f8d61ddca61ab42761d09 WatchSource:0}: Error finding container d1be76123f3405b8adcc7212870ea27124f0d24cd01f8d61ddca61ab42761d09: Status 404 returned error can't find the container with id d1be76123f3405b8adcc7212870ea27124f0d24cd01f8d61ddca61ab42761d09
Sep 04 21:54:50.741280 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.741193    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8nmrf" event={"ID":"453903df-4411-410d-a1b5-9a513da936c4","Type":"ContainerStarted","Data":"d1be76123f3405b8adcc7212870ea27124f0d24cd01f8d61ddca61ab42761d09"}
Sep 04 21:54:50.742366 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.742322    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f96gh" event={"ID":"c6b2f1f2-f011-4d5e-b80c-f15c6add8565","Type":"ContainerStarted","Data":"9a6549e52a5cdc92c47dd061dea76267fd70f1e8281c65c3da950fda67b9ad46"}
Sep 04 21:54:50.849936 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:50.849912    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x6rqt"]
Sep 04 21:54:50.852548 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:50.852526    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977ad0ad_b542_43b5_b623_21e789286a10.slice/crio-af48e1f8d93cf268447199d3cd21518ec41905637ee739ff3797c6fe6ca5b6b0 WatchSource:0}: Error finding container af48e1f8d93cf268447199d3cd21518ec41905637ee739ff3797c6fe6ca5b6b0: Status 404 returned error can't find the container with id af48e1f8d93cf268447199d3cd21518ec41905637ee739ff3797c6fe6ca5b6b0
Sep 04 21:54:51.216373 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.215291    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"]
Sep 04 21:54:51.217938 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.217913    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.220958 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.220935    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9tvmu2cq16jgd\""
Sep 04 21:54:51.221090 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.220967    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 04 21:54:51.221382 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.221361    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 04 21:54:51.222144 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.222120    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kpp4b\""
Sep 04 21:54:51.222244 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.222226    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 04 21:54:51.222375 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.222357    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 04 21:54:51.222483 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.222417    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 04 21:54:51.232424 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.231679    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"]
Sep 04 21:54:51.278776 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.278744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9f4s\" (UniqueName: \"kubernetes.io/projected/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-kube-api-access-k9f4s\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.278924 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.278790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-grpc-tls\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.278924 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.278881    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.278924 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.278912    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.279080 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.278939    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.279080 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.278964    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.279080 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.279019    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-metrics-client-ca\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.279080 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.279051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-tls\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.379713 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.379895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.379895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-metrics-client-ca\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.379895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-tls\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.379895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9f4s\" (UniqueName: \"kubernetes.io/projected/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-kube-api-access-k9f4s\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.379895 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-grpc-tls\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.380145 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379932    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.380145 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.379966    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.380551 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.380507    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-metrics-client-ca\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.384832 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.383214    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.385625 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.385552    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-grpc-tls\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.385625 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.385575    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.385835 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.385783    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.387304 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.387258    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.387725 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.387666    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-secret-thanos-querier-tls\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.391128 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.391096    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9f4s\" (UniqueName: \"kubernetes.io/projected/ce3cac16-5b9f-4202-a9b2-88470f4f15c3-kube-api-access-k9f4s\") pod \"thanos-querier-7d58d685d6-d5bg5\" (UID: \"ce3cac16-5b9f-4202-a9b2-88470f4f15c3\") " pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.531494 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.531356    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:51.750218 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.750180    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x6rqt" event={"ID":"977ad0ad-b542-43b5-b623-21e789286a10","Type":"ContainerStarted","Data":"af48e1f8d93cf268447199d3cd21518ec41905637ee739ff3797c6fe6ca5b6b0"}
Sep 04 21:54:51.753396 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.753364    2565 generic.go:356] "Generic (PLEG): container finished" podID="29723d57-fa71-4252-8517-89db77537ec3" containerID="b2f216bc59671ed904141bb7d39bb7267510113de6ea9af195ea53e8407fb508" exitCode=0
Sep 04 21:54:51.753514 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:51.753427    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6qht" event={"ID":"29723d57-fa71-4252-8517-89db77537ec3","Type":"ContainerDied","Data":"b2f216bc59671ed904141bb7d39bb7267510113de6ea9af195ea53e8407fb508"}
Sep 04 21:54:52.017528 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.016581    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"]
Sep 04 21:54:52.022149 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:52.022111    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3cac16_5b9f_4202_a9b2_88470f4f15c3.slice/crio-cd37c43d3a217f0df0891e33ef4e8f90acb651c0ee372554103f0df60d462d7c WatchSource:0}: Error finding container cd37c43d3a217f0df0891e33ef4e8f90acb651c0ee372554103f0df60d462d7c: Status 404 returned error can't find the container with id cd37c43d3a217f0df0891e33ef4e8f90acb651c0ee372554103f0df60d462d7c
Sep 04 21:54:52.622936 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.622639    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85d8b975b8-vxjbq"]
Sep 04 21:54:52.624870 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.624847    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.627634 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.627500    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 04 21:54:52.628329 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.628301    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 04 21:54:52.628329 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.628320    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 04 21:54:52.628517 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.628394    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dpkv8\""
Sep 04 21:54:52.629142 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.629124    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 04 21:54:52.629319 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.629301    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8n0k0gei1jetl\""
Sep 04 21:54:52.635725 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.635677    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85d8b975b8-vxjbq"]
Sep 04 21:54:52.697297 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697269    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.697457 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697301    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-secret-metrics-server-tls\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.697457 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697333    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-client-ca-bundle\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.697557 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-audit-log\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.697557 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-metrics-server-audit-profiles\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.697651 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697558    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-secret-metrics-server-client-certs\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.697651 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.697598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4vzc\" (UniqueName: \"kubernetes.io/projected/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-kube-api-access-f4vzc\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.760624 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.760064    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8nmrf" event={"ID":"453903df-4411-410d-a1b5-9a513da936c4","Type":"ContainerStarted","Data":"efb4e6561af19df0ec7cf835f7c6394cc4cda5127ea9705a7fa47ce85566fd6e"}
Sep 04 21:54:52.760624 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.760135    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8nmrf" event={"ID":"453903df-4411-410d-a1b5-9a513da936c4","Type":"ContainerStarted","Data":"df3b502c22b1fca7dcee0084dfb91f5e41fb00b99cb8e3df6b8cf27a09d22cab"}
Sep 04 21:54:52.763066 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.762655    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6qht" event={"ID":"29723d57-fa71-4252-8517-89db77537ec3","Type":"ContainerStarted","Data":"72dddd675611ce132f79f424ae8f84a9372ab6da7d104ab2eb692672eb86d708"}
Sep 04 21:54:52.763066 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.762715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6qht" event={"ID":"29723d57-fa71-4252-8517-89db77537ec3","Type":"ContainerStarted","Data":"c4640f51699e6d5cae394a6853f1fc82105f1f72eda95d0e8568d9f9eab391d5"}
Sep 04 21:54:52.764214 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.764191    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"cd37c43d3a217f0df0891e33ef4e8f90acb651c0ee372554103f0df60d462d7c"}
Sep 04 21:54:52.777330 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.777286    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8nmrf" podStartSLOduration=65.600592624 podStartE2EDuration="1m6.777273884s" totalImagesPullingTime="1.17668126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:52.776590931 +0000 UTC m=+67.860975777" watchObservedRunningTime="2026-09-04 21:54:52.777273884 +0000 UTC m=+67.861658728"
Sep 04 21:54:52.795014 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.794980    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k6qht" podStartSLOduration=3.078101344 podStartE2EDuration="4.794968163s" totalImagesPullingTime="1.648053363s" totalInitContainerRuntime="68.813456ms" isStatefulPod=true podCreationTimestamp="2026-09-04 21:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:52.793967941 +0000 UTC m=+67.878352786" watchObservedRunningTime="2026-09-04 21:54:52.794968163 +0000 UTC m=+67.879353008"
Sep 04 21:54:52.798802 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.798780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-metrics-server-audit-profiles\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.798907 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.798806    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-secret-metrics-server-client-certs\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.798907 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.798825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4vzc\" (UniqueName: \"kubernetes.io/projected/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-kube-api-access-f4vzc\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.799019 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.798994    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.799071 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.799031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-secret-metrics-server-tls\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.799119 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.799091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-client-ca-bundle\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.799175 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.799151    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-audit-log\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.799887 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.799514    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-audit-log\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.799979 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.799884    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.800141 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.800119    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-metrics-server-audit-profiles\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.801979 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.801957    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-secret-metrics-server-tls\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.802153 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.802130    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-client-ca-bundle\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.802214 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.802154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-secret-metrics-server-client-certs\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.808711 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.808672    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4vzc\" (UniqueName: \"kubernetes.io/projected/cd2dab44-2ef9-4387-aea4-9460ad57b7c3-kube-api-access-f4vzc\") pod \"metrics-server-85d8b975b8-vxjbq\" (UID: \"cd2dab44-2ef9-4387-aea4-9460ad57b7c3\") " pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.939032 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.938694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:54:52.990209 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.990173    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"]
Sep 04 21:54:52.994719 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.993472    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:52.996533 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.996512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"]
Sep 04 21:54:52.998492 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.998225    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nfjzj\""
Sep 04 21:54:52.998492 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:52.998329    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 04 21:54:53.102082 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.102046    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm2nx\" (UniqueName: \"kubernetes.io/projected/4c993cc6-0448-44ab-9dbb-aee44f55e7fd-kube-api-access-lm2nx\") pod \"monitoring-plugin-85bbbbbf4-r6x9k\" (UID: \"4c993cc6-0448-44ab-9dbb-aee44f55e7fd\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.102245 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.102129    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4c993cc6-0448-44ab-9dbb-aee44f55e7fd-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r6x9k\" (UID: \"4c993cc6-0448-44ab-9dbb-aee44f55e7fd\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.203540 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.203450    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm2nx\" (UniqueName: \"kubernetes.io/projected/4c993cc6-0448-44ab-9dbb-aee44f55e7fd-kube-api-access-lm2nx\") pod \"monitoring-plugin-85bbbbbf4-r6x9k\" (UID: \"4c993cc6-0448-44ab-9dbb-aee44f55e7fd\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.203694 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.203558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4c993cc6-0448-44ab-9dbb-aee44f55e7fd-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r6x9k\" (UID: \"4c993cc6-0448-44ab-9dbb-aee44f55e7fd\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.206866 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.206815    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4c993cc6-0448-44ab-9dbb-aee44f55e7fd-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r6x9k\" (UID: \"4c993cc6-0448-44ab-9dbb-aee44f55e7fd\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.213121 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.213078    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm2nx\" (UniqueName: \"kubernetes.io/projected/4c993cc6-0448-44ab-9dbb-aee44f55e7fd-kube-api-access-lm2nx\") pod \"monitoring-plugin-85bbbbbf4-r6x9k\" (UID: \"4c993cc6-0448-44ab-9dbb-aee44f55e7fd\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.238316 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.238055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85d8b975b8-vxjbq"]
Sep 04 21:54:53.308942 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.308645    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:53.454703 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.454619    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-597bf46569-crsd2"]
Sep 04 21:54:53.457100 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.457056    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.460120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.459463    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 04 21:54:53.460120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.459621    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 04 21:54:53.460120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.459464    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 04 21:54:53.460120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.459839    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 04 21:54:53.460120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.459922    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 04 21:54:53.460120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.460008    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fjwxq\""
Sep 04 21:54:53.464835 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.464809    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 04 21:54:53.473184 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.473160    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-597bf46569-crsd2"]
Sep 04 21:54:53.505762 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.505728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-serving-certs-ca-bundle\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.505915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.505777    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-federate-client-tls\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.505915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.505811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.505915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.505900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-metrics-client-ca\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.506051 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.505937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjd6q\" (UniqueName: \"kubernetes.io/projected/e24d7847-6ea7-46a8-965c-0a07924e0a19-kube-api-access-wjd6q\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.506051 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.505973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.506051 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.506002    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-secret-telemeter-client\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.506051 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.506027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-telemeter-client-tls\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607014    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjd6q\" (UniqueName: \"kubernetes.io/projected/e24d7847-6ea7-46a8-965c-0a07924e0a19-kube-api-access-wjd6q\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607086    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607124    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-secret-telemeter-client\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607149    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-telemeter-client-tls\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607193    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-serving-certs-ca-bundle\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607224    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-federate-client-tls\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607252    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.607290    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-metrics-client-ca\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.608106    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-metrics-client-ca\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.608726 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.608685    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-serving-certs-ca-bundle\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.609574 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.609417    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24d7847-6ea7-46a8-965c-0a07924e0a19-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.612376 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.612301    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-federate-client-tls\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.612376 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.612360    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-secret-telemeter-client\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.612548 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.612401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-telemeter-client-tls\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.614787 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.614742    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e24d7847-6ea7-46a8-965c-0a07924e0a19-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.618925 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.618889    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjd6q\" (UniqueName: \"kubernetes.io/projected/e24d7847-6ea7-46a8-965c-0a07924e0a19-kube-api-access-wjd6q\") pod \"telemeter-client-597bf46569-crsd2\" (UID: \"e24d7847-6ea7-46a8-965c-0a07924e0a19\") " pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:53.770815 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:53.770724    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2"
Sep 04 21:54:54.421855 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.419987    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 21:54:54.424888 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.424859    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.427070 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427047    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 04 21:54:54.427249 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427228    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fkm0dodv6n2k7\""
Sep 04 21:54:54.427320 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427265    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 04 21:54:54.427320 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427298    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 04 21:54:54.428240 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 04 21:54:54.428240 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427856    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 04 21:54:54.428240 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427877    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 04 21:54:54.428240 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.427895    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 04 21:54:54.428240 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.428035    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 04 21:54:54.428240 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.428106    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 04 21:54:54.428907 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.428415    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l7lpf\""
Sep 04 21:54:54.428907 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.428589    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 04 21:54:54.429405 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.429098    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 04 21:54:54.429405 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.429160    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 04 21:54:54.429405 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.429163    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 04 21:54:54.431109 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.431094    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 04 21:54:54.439868 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.439838    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 21:54:54.515560 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515144    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.515560 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515285    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.515560 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.515560 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.515560 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-config\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.515560 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-config-out\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-web-config\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515877    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515922    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d4wz\" (UniqueName: \"kubernetes.io/projected/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-kube-api-access-5d4wz\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.515977    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516017 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.516013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516589 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.516049    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516589 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.516076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516589 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.516090    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.516589 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.516112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.616753 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.616913 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.616913 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.616913 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.616913 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-config\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.616913 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-config-out\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.617151 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-web-config\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.617151 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616947    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.617151 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.616973    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.617680 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.617647    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.617792 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.617731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.617792 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.617762    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618250 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618366 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618366 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.617769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618366 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618320    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d4wz\" (UniqueName: \"kubernetes.io/projected/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-kube-api-access-5d4wz\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618465    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618512 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618931 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618514    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.618931 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.618546    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.619199 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.619172    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.621572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.620518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.621572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.620693    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.621572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.620851    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.621572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.620979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.621572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.621053    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.621572 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.621328    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-web-config\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.621893    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.621980    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-config-out\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.622241    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.622252    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-config\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.622253    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.623023    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.623990 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.623944    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.624418 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.624398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.624637 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.624615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.633506 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.633477    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d4wz\" (UniqueName: \"kubernetes.io/projected/9ca58a9a-53bc-4ba8-84d8-23e112e1f86f-kube-api-access-5d4wz\") pod \"prometheus-k8s-0\" (UID: \"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:54.738737 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:54.738649    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:54:55.803027 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:55.802982    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2dab44_2ef9_4387_aea4_9460ad57b7c3.slice/crio-87f994434514308da9bf59c1f77c79a4000b54bd56345f0ea070fa371b1057d8 WatchSource:0}: Error finding container 87f994434514308da9bf59c1f77c79a4000b54bd56345f0ea070fa371b1057d8: Status 404 returned error can't find the container with id 87f994434514308da9bf59c1f77c79a4000b54bd56345f0ea070fa371b1057d8
Sep 04 21:54:56.630373 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.629018    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"]
Sep 04 21:54:56.633468 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:56.633435    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c993cc6_0448_44ab_9dbb_aee44f55e7fd.slice/crio-78d7da54aea66993f05e07cee19e3f833db3c5f64180eddd815d8270f93ff4e4 WatchSource:0}: Error finding container 78d7da54aea66993f05e07cee19e3f833db3c5f64180eddd815d8270f93ff4e4: Status 404 returned error can't find the container with id 78d7da54aea66993f05e07cee19e3f833db3c5f64180eddd815d8270f93ff4e4
Sep 04 21:54:56.653589 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.653564    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 21:54:56.657095 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:56.657068    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca58a9a_53bc_4ba8_84d8_23e112e1f86f.slice/crio-4a265a8b1718478aeafbfcbd957f7a315f948e317c2a16eb6c054dd9f08b5d8d WatchSource:0}: Error finding container 4a265a8b1718478aeafbfcbd957f7a315f948e317c2a16eb6c054dd9f08b5d8d: Status 404 returned error can't find the container with id 4a265a8b1718478aeafbfcbd957f7a315f948e317c2a16eb6c054dd9f08b5d8d
Sep 04 21:54:56.661937 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.661862    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-597bf46569-crsd2"]
Sep 04 21:54:56.665224 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:54:56.665202    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24d7847_6ea7_46a8_965c_0a07924e0a19.slice/crio-36e4b58cb3d35e8f5753833f77b2f058935b3f1a37be1855cca3defb46a68547 WatchSource:0}: Error finding container 36e4b58cb3d35e8f5753833f77b2f058935b3f1a37be1855cca3defb46a68547: Status 404 returned error can't find the container with id 36e4b58cb3d35e8f5753833f77b2f058935b3f1a37be1855cca3defb46a68547
Sep 04 21:54:56.784780 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.784313    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x6rqt" event={"ID":"977ad0ad-b542-43b5-b623-21e789286a10","Type":"ContainerStarted","Data":"eca02a830d51c35e9cbaebe85118252360e5c01566d73b9435e0c51b13920633"}
Sep 04 21:54:56.784780 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.784557    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:54:56.787400 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.787355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f96gh" event={"ID":"c6b2f1f2-f011-4d5e-b80c-f15c6add8565","Type":"ContainerStarted","Data":"bb33a2f749644a73fe4b5190fb940ce0dcdc672804b0308d94d4e31a8682987f"}
Sep 04 21:54:56.789234 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.789203    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k" event={"ID":"4c993cc6-0448-44ab-9dbb-aee44f55e7fd","Type":"ContainerStarted","Data":"78d7da54aea66993f05e07cee19e3f833db3c5f64180eddd815d8270f93ff4e4"}
Sep 04 21:54:56.790579 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.790556    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq" event={"ID":"cd2dab44-2ef9-4387-aea4-9460ad57b7c3","Type":"ContainerStarted","Data":"87f994434514308da9bf59c1f77c79a4000b54bd56345f0ea070fa371b1057d8"}
Sep 04 21:54:56.792813 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.792789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"c2a5f938cef5bf16f403149cac5f92454866d93e43279f819b9a9d0f663afe1b"}
Sep 04 21:54:56.792915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.792863    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"2795e56a5122b85110fb42ed960597af3043ca8b3486ea06c1efcaea928e651c"}
Sep 04 21:54:56.792915 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.792904    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"76ee48bfa9f4a0dabe06bdad860477682583d045d41e752da26ff37f397222db"}
Sep 04 21:54:56.794207 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.794184    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"4a265a8b1718478aeafbfcbd957f7a315f948e317c2a16eb6c054dd9f08b5d8d"}
Sep 04 21:54:56.795329 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.795305    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2" event={"ID":"e24d7847-6ea7-46a8-965c-0a07924e0a19","Type":"ContainerStarted","Data":"36e4b58cb3d35e8f5753833f77b2f058935b3f1a37be1855cca3defb46a68547"}
Sep 04 21:54:56.801024 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.800979    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x6rqt" podStartSLOduration=65.293207711 podStartE2EDuration="1m10.800967629s" totalImagesPullingTime="5.507759918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:56.799957452 +0000 UTC m=+71.884342297" watchObservedRunningTime="2026-09-04 21:54:56.800967629 +0000 UTC m=+71.885352473"
Sep 04 21:54:56.818624 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:56.818583    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f96gh" podStartSLOduration=65.048407316 podStartE2EDuration="1m10.818570663s" totalImagesPullingTime="5.770163347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:56.817781183 +0000 UTC m=+71.902166029" watchObservedRunningTime="2026-09-04 21:54:56.818570663 +0000 UTC m=+71.902955507"
Sep 04 21:54:57.803878 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:57.803391    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"d4a9d3af5627820b0bc28dfcee531c4b9a294b28c6dadba53f05a1b43c26f26e"}
Sep 04 21:54:57.806775 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:57.806220    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq" event={"ID":"cd2dab44-2ef9-4387-aea4-9460ad57b7c3","Type":"ContainerStarted","Data":"8f3892c7d715acb46dbdd071e6cb3c3bf3fb2312b6fde86b011f7fecddb1cb8e"}
Sep 04 21:54:57.844940 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:57.844843    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq" podStartSLOduration=4.000028878 podStartE2EDuration="5.844823843s" totalImagesPullingTime="1.844794965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:57.843588177 +0000 UTC m=+72.927973022" watchObservedRunningTime="2026-09-04 21:54:57.844823843 +0000 UTC m=+72.929208689"
Sep 04 21:54:58.814009 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.813921    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"6e7106b1d1d7c9bc494c5ab97f046de3b9da47001f5cb27fffce4df6eaffd67e"}
Sep 04 21:54:58.814163 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.814007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"4965f2885f4eab11a52966b664bbd12e331e3329692612f9b9aa7e337c2531cb"}
Sep 04 21:54:58.814163 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.814053    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" event={"ID":"ce3cac16-5b9f-4202-a9b2-88470f4f15c3","Type":"ContainerStarted","Data":"03aa1bba7225b4b9157f093b9084efe7fd84ead99d8e30e221dea2c20cead257"}
Sep 04 21:54:58.814163 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.814098    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:54:58.815662 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.815634    2565 generic.go:356] "Generic (PLEG): container finished" podID="9ca58a9a-53bc-4ba8-84d8-23e112e1f86f" containerID="d4a9d3af5627820b0bc28dfcee531c4b9a294b28c6dadba53f05a1b43c26f26e" exitCode=0
Sep 04 21:54:58.815785 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.815725    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerDied","Data":"d4a9d3af5627820b0bc28dfcee531c4b9a294b28c6dadba53f05a1b43c26f26e"}
Sep 04 21:54:58.817886 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.817463    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k" event={"ID":"4c993cc6-0448-44ab-9dbb-aee44f55e7fd","Type":"ContainerStarted","Data":"8cc08f5c5ebcebe1dd1418fe8407e1f9f77840fa8e6a31f6fac3e8d8b91e6f8a"}
Sep 04 21:54:58.817886 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.817520    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:58.823131 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.823111    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k"
Sep 04 21:54:58.838103 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.838062    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5" podStartSLOduration=2.220726263 podStartE2EDuration="7.838048164s" totalImagesPullingTime="5.617321901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:58.837078261 +0000 UTC m=+73.921463106" watchObservedRunningTime="2026-09-04 21:54:58.838048164 +0000 UTC m=+73.922433008"
Sep 04 21:54:58.884587 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:58.884540    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r6x9k" podStartSLOduration=5.090302802 podStartE2EDuration="6.884524197s" totalImagesPullingTime="1.794221395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:58.883357197 +0000 UTC m=+73.967742041" watchObservedRunningTime="2026-09-04 21:54:58.884524197 +0000 UTC m=+73.968909040"
Sep 04 21:54:59.824418 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:59.824376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2" event={"ID":"e24d7847-6ea7-46a8-965c-0a07924e0a19","Type":"ContainerStarted","Data":"9e3b38ddb6ad4a060c3776f85a57a838bcd68916afd9c59cf7d408801a7c65f4"}
Sep 04 21:54:59.824593 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:59.824448    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2" event={"ID":"e24d7847-6ea7-46a8-965c-0a07924e0a19","Type":"ContainerStarted","Data":"8b8d498dee6ab227e98635d79adfc75f1464708fd5cbf4a309f15513001554b3"}
Sep 04 21:54:59.824593 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:59.824476    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2" event={"ID":"e24d7847-6ea7-46a8-965c-0a07924e0a19","Type":"ContainerStarted","Data":"c2d1dfd1a38f1a35483ad64997b51b143772357ef835284003d155bd0acf9e2a"}
Sep 04 21:54:59.848046 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:54:59.847982    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-597bf46569-crsd2" podStartSLOduration=4.302533456 podStartE2EDuration="6.847961598s" totalImagesPullingTime="2.545428142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:59.844907956 +0000 UTC m=+74.929292828" watchObservedRunningTime="2026-09-04 21:54:59.847961598 +0000 UTC m=+74.932346444"
Sep 04 21:55:02.840844 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.840803    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"ce8b6cfc26e412b2002da77f05d02f414a9adefadb594a1df48813a197abf2c5"}
Sep 04 21:55:02.841253 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.840877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"3873d2f395585e45eca4c98f4400ddb53dde718ab933a7bb206c2beae2509d8b"}
Sep 04 21:55:02.841253 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.840911    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"cdad76c928b434985870bac4916902ad0196abe9cad76cda6f1102cf86fa7f9c"}
Sep 04 21:55:02.841253 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.840937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"92244e507ff0d5684e4412cb402921388de01f53f62e80fa8ee6764787b1faa7"}
Sep 04 21:55:02.841253 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.840962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"f7e9e9872c23b67efce12944eb7de72c8bb3654d02b1a0a54a4f9400cfa12099"}
Sep 04 21:55:02.841253 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.840992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9ca58a9a-53bc-4ba8-84d8-23e112e1f86f","Type":"ContainerStarted","Data":"393e3dc4f56c665ca1924ca9d5972aafa59e6a738fe5fab099769cacb0b30c13"}
Sep 04 21:55:02.871748 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:02.871464    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.722845933 podStartE2EDuration="8.871443255s" totalImagesPullingTime="4.119524244s" totalInitContainerRuntime="29.073078ms" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:55:02.869943145 +0000 UTC m=+77.954327990" watchObservedRunningTime="2026-09-04 21:55:02.871443255 +0000 UTC m=+77.955828102"
Sep 04 21:55:04.738757 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:04.738720    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:55:04.830622 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:04.830595    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d58d685d6-d5bg5"
Sep 04 21:55:12.939919 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:12.939799    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:55:12.939919 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:12.939866    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:55:27.809405 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:27.809375    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x6rqt"
Sep 04 21:55:32.945404 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:32.945371    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:55:32.949161 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:32.949131    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85d8b975b8-vxjbq"
Sep 04 21:55:54.739148 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:54.739096    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:55:54.758357 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:54.758304    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:55:55.051332 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:55:55.051251    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 21:57:36.639790 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.639760    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9f64q"]
Sep 04 21:57:36.642352 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.642314    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.644739 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.644722    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 04 21:57:36.644830 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.644740    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 04 21:57:36.645381 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.645362    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-p24qs\""
Sep 04 21:57:36.652334 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.652315    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9f64q"]
Sep 04 21:57:36.694600 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.694572    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e35e8ddc-38b0-4dee-9d08-334472cb3502-bound-sa-token\") pod \"cert-manager-55b4d954d5-9f64q\" (UID: \"e35e8ddc-38b0-4dee-9d08-334472cb3502\") " pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.694688 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.694616    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48sz2\" (UniqueName: \"kubernetes.io/projected/e35e8ddc-38b0-4dee-9d08-334472cb3502-kube-api-access-48sz2\") pod \"cert-manager-55b4d954d5-9f64q\" (UID: \"e35e8ddc-38b0-4dee-9d08-334472cb3502\") " pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.795164 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.795138    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e35e8ddc-38b0-4dee-9d08-334472cb3502-bound-sa-token\") pod \"cert-manager-55b4d954d5-9f64q\" (UID: \"e35e8ddc-38b0-4dee-9d08-334472cb3502\") " pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.795301 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.795184    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48sz2\" (UniqueName: \"kubernetes.io/projected/e35e8ddc-38b0-4dee-9d08-334472cb3502-kube-api-access-48sz2\") pod \"cert-manager-55b4d954d5-9f64q\" (UID: \"e35e8ddc-38b0-4dee-9d08-334472cb3502\") " pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.803282 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.803259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e35e8ddc-38b0-4dee-9d08-334472cb3502-bound-sa-token\") pod \"cert-manager-55b4d954d5-9f64q\" (UID: \"e35e8ddc-38b0-4dee-9d08-334472cb3502\") " pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.803282 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.803277    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48sz2\" (UniqueName: \"kubernetes.io/projected/e35e8ddc-38b0-4dee-9d08-334472cb3502-kube-api-access-48sz2\") pod \"cert-manager-55b4d954d5-9f64q\" (UID: \"e35e8ddc-38b0-4dee-9d08-334472cb3502\") " pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:36.969253 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:36.969163    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9f64q"
Sep 04 21:57:37.237423 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:37.237401    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9f64q"]
Sep 04 21:57:37.240015 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:57:37.239987    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode35e8ddc_38b0_4dee_9d08_334472cb3502.slice/crio-c53d5678b53d42b63a00b2a6b90983de2cdca5d5c1076aa1a57736280d0e6974 WatchSource:0}: Error finding container c53d5678b53d42b63a00b2a6b90983de2cdca5d5c1076aa1a57736280d0e6974: Status 404 returned error can't find the container with id c53d5678b53d42b63a00b2a6b90983de2cdca5d5c1076aa1a57736280d0e6974
Sep 04 21:57:37.406834 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:37.406801    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9f64q" event={"ID":"e35e8ddc-38b0-4dee-9d08-334472cb3502","Type":"ContainerStarted","Data":"c53d5678b53d42b63a00b2a6b90983de2cdca5d5c1076aa1a57736280d0e6974"}
Sep 04 21:57:40.420656 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:40.420619    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9f64q" event={"ID":"e35e8ddc-38b0-4dee-9d08-334472cb3502","Type":"ContainerStarted","Data":"110954b91a18cbcf1d3df3b61ae87b66c8d3d3d8637a9910be4416fb808dfaff"}
Sep 04 21:57:40.436095 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:40.436049    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9f64q" podStartSLOduration=1.6096304080000001 podStartE2EDuration="4.436033684s" totalImagesPullingTime="2.826403276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:57:40.434781664 +0000 UTC m=+235.519166508" watchObservedRunningTime="2026-09-04 21:57:40.436033684 +0000 UTC m=+235.520418530"
Sep 04 21:57:44.303424 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.303384    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"]
Sep 04 21:57:44.305541 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.305526    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.307862 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.307837    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 04 21:57:44.307862 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.307854    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:57:44.308602 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.308588    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kk5g7\""
Sep 04 21:57:44.314936 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.314916    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"]
Sep 04 21:57:44.357957 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.357927    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr2x5\" (UniqueName: \"kubernetes.io/projected/c8f5181c-dd72-4add-8068-6e12c5f03a98-kube-api-access-dr2x5\") pod \"openshift-lws-operator-fd8ccff4c-t5w88\" (UID: \"c8f5181c-dd72-4add-8068-6e12c5f03a98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.358105 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.357986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8f5181c-dd72-4add-8068-6e12c5f03a98-tmp\") pod \"openshift-lws-operator-fd8ccff4c-t5w88\" (UID: \"c8f5181c-dd72-4add-8068-6e12c5f03a98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.459364 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.459321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr2x5\" (UniqueName: \"kubernetes.io/projected/c8f5181c-dd72-4add-8068-6e12c5f03a98-kube-api-access-dr2x5\") pod \"openshift-lws-operator-fd8ccff4c-t5w88\" (UID: \"c8f5181c-dd72-4add-8068-6e12c5f03a98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.459550 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.459516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8f5181c-dd72-4add-8068-6e12c5f03a98-tmp\") pod \"openshift-lws-operator-fd8ccff4c-t5w88\" (UID: \"c8f5181c-dd72-4add-8068-6e12c5f03a98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.459924 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.459905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8f5181c-dd72-4add-8068-6e12c5f03a98-tmp\") pod \"openshift-lws-operator-fd8ccff4c-t5w88\" (UID: \"c8f5181c-dd72-4add-8068-6e12c5f03a98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.470211 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.470186    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr2x5\" (UniqueName: \"kubernetes.io/projected/c8f5181c-dd72-4add-8068-6e12c5f03a98-kube-api-access-dr2x5\") pod \"openshift-lws-operator-fd8ccff4c-t5w88\" (UID: \"c8f5181c-dd72-4add-8068-6e12c5f03a98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.614662 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.614637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"
Sep 04 21:57:44.885591 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:44.885520    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88"]
Sep 04 21:57:44.890864 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:57:44.890838    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8f5181c_dd72_4add_8068_6e12c5f03a98.slice/crio-4aa0ba9e9d5b53bb04481f82d51cba05bec886d346059d34eb191da6eaaa674c WatchSource:0}: Error finding container 4aa0ba9e9d5b53bb04481f82d51cba05bec886d346059d34eb191da6eaaa674c: Status 404 returned error can't find the container with id 4aa0ba9e9d5b53bb04481f82d51cba05bec886d346059d34eb191da6eaaa674c
Sep 04 21:57:45.439102 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:45.439065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88" event={"ID":"c8f5181c-dd72-4add-8068-6e12c5f03a98","Type":"ContainerStarted","Data":"4aa0ba9e9d5b53bb04481f82d51cba05bec886d346059d34eb191da6eaaa674c"}
Sep 04 21:57:48.453361 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:48.453308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88" event={"ID":"c8f5181c-dd72-4add-8068-6e12c5f03a98","Type":"ContainerStarted","Data":"cb46dc5dbe094ab692f5187b8396d65bb1dc354018f148a8de764cff630092ef"}
Sep 04 21:57:48.474540 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:57:48.474496    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t5w88" podStartSLOduration=1.592376583 podStartE2EDuration="4.474481188s" totalImagesPullingTime="2.882104605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:57:48.473233253 +0000 UTC m=+243.557618098" watchObservedRunningTime="2026-09-04 21:57:48.474481188 +0000 UTC m=+243.558866031"
Sep 04 21:58:03.857827 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.857793    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"]
Sep 04 21:58:03.862475 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.862455    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:03.868120 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.868080    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 04 21:58:03.868276 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.868195    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 04 21:58:03.868276 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.868230    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 04 21:58:03.869112 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.869089    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 04 21:58:03.869112 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.869094    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jrmss\""
Sep 04 21:58:03.882598 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.882574    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"]
Sep 04 21:58:03.914015 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.913980    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2djnr\" (UniqueName: \"kubernetes.io/projected/704c64d7-cdd6-4574-9f9f-90bcb7be3144-kube-api-access-2djnr\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:03.914193 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.914025    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/704c64d7-cdd6-4574-9f9f-90bcb7be3144-tmp\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:03.914193 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.914054    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/704c64d7-cdd6-4574-9f9f-90bcb7be3144-webhook-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:03.914193 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.914133    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/704c64d7-cdd6-4574-9f9f-90bcb7be3144-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:03.914193 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:03.914179    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/704c64d7-cdd6-4574-9f9f-90bcb7be3144-manifests\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.015002 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.014962    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/704c64d7-cdd6-4574-9f9f-90bcb7be3144-webhook-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.015002 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.015004    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/704c64d7-cdd6-4574-9f9f-90bcb7be3144-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.015233 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.015030    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/704c64d7-cdd6-4574-9f9f-90bcb7be3144-manifests\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.015233 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.015083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2djnr\" (UniqueName: \"kubernetes.io/projected/704c64d7-cdd6-4574-9f9f-90bcb7be3144-kube-api-access-2djnr\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.015233 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.015108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/704c64d7-cdd6-4574-9f9f-90bcb7be3144-tmp\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.015569 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.015543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/704c64d7-cdd6-4574-9f9f-90bcb7be3144-manifests\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.017401 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.017379    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/704c64d7-cdd6-4574-9f9f-90bcb7be3144-tmp\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.017657 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.017639    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/704c64d7-cdd6-4574-9f9f-90bcb7be3144-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.017715 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.017697    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/704c64d7-cdd6-4574-9f9f-90bcb7be3144-webhook-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.023617 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.023594    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2djnr\" (UniqueName: \"kubernetes.io/projected/704c64d7-cdd6-4574-9f9f-90bcb7be3144-kube-api-access-2djnr\") pod \"opendatahub-operator-controller-manager-764b98687b-5hmtg\" (UID: \"704c64d7-cdd6-4574-9f9f-90bcb7be3144\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.173026 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.172939    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:04.450909 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.450884    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"]
Sep 04 21:58:04.453591 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:58:04.453553    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704c64d7_cdd6_4574_9f9f_90bcb7be3144.slice/crio-69711e2931c33567d4084fe27e0fae5d6dcdcc376898acd56d8a3b383086369a WatchSource:0}: Error finding container 69711e2931c33567d4084fe27e0fae5d6dcdcc376898acd56d8a3b383086369a: Status 404 returned error can't find the container with id 69711e2931c33567d4084fe27e0fae5d6dcdcc376898acd56d8a3b383086369a
Sep 04 21:58:04.512907 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:04.512879    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg" event={"ID":"704c64d7-cdd6-4574-9f9f-90bcb7be3144","Type":"ContainerStarted","Data":"69711e2931c33567d4084fe27e0fae5d6dcdcc376898acd56d8a3b383086369a"}
Sep 04 21:58:08.533189 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:08.533148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg" event={"ID":"704c64d7-cdd6-4574-9f9f-90bcb7be3144","Type":"ContainerStarted","Data":"2972d16eee69f9a111c54dad8597283e7c4c94001d970d44f4592f774f607a4a"}
Sep 04 21:58:09.539670 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:09.539632    2565 generic.go:356] "Generic (PLEG): container finished" podID="704c64d7-cdd6-4574-9f9f-90bcb7be3144" containerID="2972d16eee69f9a111c54dad8597283e7c4c94001d970d44f4592f774f607a4a" exitCode=0
Sep 04 21:58:09.540139 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:09.539679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg" event={"ID":"704c64d7-cdd6-4574-9f9f-90bcb7be3144","Type":"ContainerDied","Data":"2972d16eee69f9a111c54dad8597283e7c4c94001d970d44f4592f774f607a4a"}
Sep 04 21:58:10.546065 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:10.545968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg" event={"ID":"704c64d7-cdd6-4574-9f9f-90bcb7be3144","Type":"ContainerStarted","Data":"99c49fb71b2c3411fcda6b348176d89aadd55615b954ba78ff64bea2c76d547c"}
Sep 04 21:58:10.546467 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:10.546080    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:10.568835 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:10.568785    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg" podStartSLOduration=2.890896749 podStartE2EDuration="7.5687692s" totalImagesPullingTime="3.640362041s" totalInitContainerRuntime="1.03751041s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:10.568072233 +0000 UTC m=+265.652457078" watchObservedRunningTime="2026-09-04 21:58:10.5687692 +0000 UTC m=+265.653154043"
Sep 04 21:58:21.553214 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:21.553187    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-5hmtg"
Sep 04 21:58:23.874064 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.874033    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"]
Sep 04 21:58:23.882630 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.882610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.886011 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.885983    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 04 21:58:23.886139 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.886041    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ms7cd\""
Sep 04 21:58:23.886139 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.886066    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 04 21:58:23.886139 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.885992    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 04 21:58:23.887593 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.887571    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"]
Sep 04 21:58:23.893329 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.893308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshb8\" (UniqueName: \"kubernetes.io/projected/cf36392c-3964-462d-970c-12131932dc58-kube-api-access-tshb8\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.893435 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.893356    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf36392c-3964-462d-970c-12131932dc58-cert\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.893435 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.893375    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf36392c-3964-462d-970c-12131932dc58-metrics-cert\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.893435 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.893427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf36392c-3964-462d-970c-12131932dc58-manager-config\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.993942 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.993907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf36392c-3964-462d-970c-12131932dc58-manager-config\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.994102 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.993963    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshb8\" (UniqueName: \"kubernetes.io/projected/cf36392c-3964-462d-970c-12131932dc58-kube-api-access-tshb8\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.994102 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.993988    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf36392c-3964-462d-970c-12131932dc58-cert\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.994102 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.994073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf36392c-3964-462d-970c-12131932dc58-metrics-cert\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.994569 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.994541    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf36392c-3964-462d-970c-12131932dc58-manager-config\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.996636 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.996615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf36392c-3964-462d-970c-12131932dc58-cert\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:23.996754 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:23.996655    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf36392c-3964-462d-970c-12131932dc58-metrics-cert\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:24.001707 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:24.001684    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshb8\" (UniqueName: \"kubernetes.io/projected/cf36392c-3964-462d-970c-12131932dc58-kube-api-access-tshb8\") pod \"lws-controller-manager-587bddd5d4-qzh96\" (UID: \"cf36392c-3964-462d-970c-12131932dc58\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:24.192558 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:24.192475    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:24.465164 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:24.465094    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"]
Sep 04 21:58:24.468131 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:58:24.468107    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf36392c_3964_462d_970c_12131932dc58.slice/crio-8c5a812f9af1cafdc7af7e96ab241a9af6a8d819ff29302c682a1c5079cb0828 WatchSource:0}: Error finding container 8c5a812f9af1cafdc7af7e96ab241a9af6a8d819ff29302c682a1c5079cb0828: Status 404 returned error can't find the container with id 8c5a812f9af1cafdc7af7e96ab241a9af6a8d819ff29302c682a1c5079cb0828
Sep 04 21:58:24.604534 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:24.604500    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96" event={"ID":"cf36392c-3964-462d-970c-12131932dc58","Type":"ContainerStarted","Data":"8c5a812f9af1cafdc7af7e96ab241a9af6a8d819ff29302c682a1c5079cb0828"}
Sep 04 21:58:26.614873 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:26.614834    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96" event={"ID":"cf36392c-3964-462d-970c-12131932dc58","Type":"ContainerStarted","Data":"2969012f6670646a2d79c1e8a066ff228aa637ee4e5ab5a1310a749c2535f4b7"}
Sep 04 21:58:26.615279 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:26.614970    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:26.632012 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:26.631964    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96" podStartSLOduration=1.996601332 podStartE2EDuration="3.631950792s" totalImagesPullingTime="1.63534946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:26.62990685 +0000 UTC m=+281.714291694" watchObservedRunningTime="2026-09-04 21:58:26.631950792 +0000 UTC m=+281.716335637"
Sep 04 21:58:33.988060 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:33.988028    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"]
Sep 04 21:58:33.991466 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:33.991450    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:33.993733 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:33.993711    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 04 21:58:33.993910 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:33.993890    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 04 21:58:33.993980 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:33.993931    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 04 21:58:33.994035 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:33.994009    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bn6sv\""
Sep 04 21:58:34.001933 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.001917    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 04 21:58:34.005321 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.005297    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"]
Sep 04 21:58:34.073856 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.073826    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ce2525-b7c3-41a0-82ee-3226a31ff455-tmp\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.074079 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.073876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ce2525-b7c3-41a0-82ee-3226a31ff455-tls-certs\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.074079 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.073953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68phb\" (UniqueName: \"kubernetes.io/projected/c7ce2525-b7c3-41a0-82ee-3226a31ff455-kube-api-access-68phb\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.174977 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.174943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68phb\" (UniqueName: \"kubernetes.io/projected/c7ce2525-b7c3-41a0-82ee-3226a31ff455-kube-api-access-68phb\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.175140 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.175017    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ce2525-b7c3-41a0-82ee-3226a31ff455-tmp\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.175140 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.175070    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ce2525-b7c3-41a0-82ee-3226a31ff455-tls-certs\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.177368 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.177313    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ce2525-b7c3-41a0-82ee-3226a31ff455-tmp\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.177481 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.177430    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ce2525-b7c3-41a0-82ee-3226a31ff455-tls-certs\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.199526 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.199498    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68phb\" (UniqueName: \"kubernetes.io/projected/c7ce2525-b7c3-41a0-82ee-3226a31ff455-kube-api-access-68phb\") pod \"kube-auth-proxy-85db8bc976-sc7bd\" (UID: \"c7ce2525-b7c3-41a0-82ee-3226a31ff455\") " pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.301494 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.301422    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"
Sep 04 21:58:34.571756 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.571733    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd"]
Sep 04 21:58:34.574270 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:58:34.574246    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ce2525_b7c3_41a0_82ee_3226a31ff455.slice/crio-c6494c77ff1c941c1ad97f20869562347c21f8584b38fc9e961cebf04b75e03a WatchSource:0}: Error finding container c6494c77ff1c941c1ad97f20869562347c21f8584b38fc9e961cebf04b75e03a: Status 404 returned error can't find the container with id c6494c77ff1c941c1ad97f20869562347c21f8584b38fc9e961cebf04b75e03a
Sep 04 21:58:34.648398 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:34.648369    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd" event={"ID":"c7ce2525-b7c3-41a0-82ee-3226a31ff455","Type":"ContainerStarted","Data":"c6494c77ff1c941c1ad97f20869562347c21f8584b38fc9e961cebf04b75e03a"}
Sep 04 21:58:37.620744 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:37.620717    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qzh96"
Sep 04 21:58:37.665019 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:37.664986    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd" event={"ID":"c7ce2525-b7c3-41a0-82ee-3226a31ff455","Type":"ContainerStarted","Data":"fc41f5123ddec529198211c969a619d076d7367f37c85faba7843b2cf80f5981"}
Sep 04 21:58:37.681995 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:37.681936    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85db8bc976-sc7bd" podStartSLOduration=1.700722561 podStartE2EDuration="4.681921446s" totalImagesPullingTime="2.981198885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:37.680174664 +0000 UTC m=+292.764559509" watchObservedRunningTime="2026-09-04 21:58:37.681921446 +0000 UTC m=+292.766306289"
Sep 04 21:58:45.302822 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.302791    2565 kubelet.go:1648] "Image garbage collection succeeded"
Sep 04 21:58:45.773873 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.773845    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"]
Sep 04 21:58:45.776732 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.776710    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:45.779735 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.779711    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 04 21:58:45.779735 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.779732    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7j89r\""
Sep 04 21:58:45.792231 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.792205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"]
Sep 04 21:58:45.874619 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.874588    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcdrd\" (UniqueName: \"kubernetes.io/projected/06fa6853-e605-4a9a-94ca-a3f8d5218f95-kube-api-access-vcdrd\") pod \"llmisvc-controller-manager-777c6bd7d5-vvz9x\" (UID: \"06fa6853-e605-4a9a-94ca-a3f8d5218f95\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:45.874782 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.874656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06fa6853-e605-4a9a-94ca-a3f8d5218f95-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vvz9x\" (UID: \"06fa6853-e605-4a9a-94ca-a3f8d5218f95\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:45.975643 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.975610    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06fa6853-e605-4a9a-94ca-a3f8d5218f95-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vvz9x\" (UID: \"06fa6853-e605-4a9a-94ca-a3f8d5218f95\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:45.975805 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.975689    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcdrd\" (UniqueName: \"kubernetes.io/projected/06fa6853-e605-4a9a-94ca-a3f8d5218f95-kube-api-access-vcdrd\") pod \"llmisvc-controller-manager-777c6bd7d5-vvz9x\" (UID: \"06fa6853-e605-4a9a-94ca-a3f8d5218f95\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:45.978205 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.978173    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06fa6853-e605-4a9a-94ca-a3f8d5218f95-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vvz9x\" (UID: \"06fa6853-e605-4a9a-94ca-a3f8d5218f95\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:45.984730 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:45.984706    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcdrd\" (UniqueName: \"kubernetes.io/projected/06fa6853-e605-4a9a-94ca-a3f8d5218f95-kube-api-access-vcdrd\") pod \"llmisvc-controller-manager-777c6bd7d5-vvz9x\" (UID: \"06fa6853-e605-4a9a-94ca-a3f8d5218f95\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:46.085996 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:46.085971    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:46.355466 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:46.355441    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"]
Sep 04 21:58:46.357917 ip-10-0-133-119 kubenswrapper[2565]: W0904 21:58:46.357887    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod06fa6853_e605_4a9a_94ca_a3f8d5218f95.slice/crio-06cd4723b2959c17d4c6c7cbf5b0c303455da210d76cac3d36e4f4bed1d2ce6e WatchSource:0}: Error finding container 06cd4723b2959c17d4c6c7cbf5b0c303455da210d76cac3d36e4f4bed1d2ce6e: Status 404 returned error can't find the container with id 06cd4723b2959c17d4c6c7cbf5b0c303455da210d76cac3d36e4f4bed1d2ce6e
Sep 04 21:58:46.359300 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:46.359284    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 21:58:46.705916 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:46.705839    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x" event={"ID":"06fa6853-e605-4a9a-94ca-a3f8d5218f95","Type":"ContainerStarted","Data":"06cd4723b2959c17d4c6c7cbf5b0c303455da210d76cac3d36e4f4bed1d2ce6e"}
Sep 04 21:58:49.723583 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:49.723541    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x" event={"ID":"06fa6853-e605-4a9a-94ca-a3f8d5218f95","Type":"ContainerStarted","Data":"6c2d81593187837bbff6361262d9511e6293789c0a6429393887ddee918637d1"}
Sep 04 21:58:49.723967 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:49.723647    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 21:58:49.742487 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:58:49.742441    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x" podStartSLOduration=2.194587001 podStartE2EDuration="4.742427762s" totalImagesPullingTime="2.547840761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:49.741013411 +0000 UTC m=+304.825398257" watchObservedRunningTime="2026-09-04 21:58:49.742427762 +0000 UTC m=+304.826812607"
Sep 04 21:59:20.729954 ip-10-0-133-119 kubenswrapper[2565]: I0904 21:59:20.729920    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vvz9x"
Sep 04 22:01:06.533383 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.533295    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:06.536477 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.536462    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.538847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.538829    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 22:01:06.539036 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.539023    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 04 22:01:06.539109 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.539077    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 22:01:06.539592 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.539576    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mlfpp\""
Sep 04 22:01:06.547715 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.547696    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:06.561398 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.561376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sv6k\" (UniqueName: \"kubernetes.io/projected/b5868856-2174-44b5-9c94-410d0da236d6-kube-api-access-8sv6k\") pod \"limitador-limitador-859c4d96d6-lnqpp\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.561495 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.561407    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5868856-2174-44b5-9c94-410d0da236d6-config-file\") pod \"limitador-limitador-859c4d96d6-lnqpp\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.622718 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.622683    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:06.662028 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.661995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sv6k\" (UniqueName: \"kubernetes.io/projected/b5868856-2174-44b5-9c94-410d0da236d6-kube-api-access-8sv6k\") pod \"limitador-limitador-859c4d96d6-lnqpp\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.662028 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.662031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5868856-2174-44b5-9c94-410d0da236d6-config-file\") pod \"limitador-limitador-859c4d96d6-lnqpp\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.662644 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.662626    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5868856-2174-44b5-9c94-410d0da236d6-config-file\") pod \"limitador-limitador-859c4d96d6-lnqpp\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.670239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.670215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sv6k\" (UniqueName: \"kubernetes.io/projected/b5868856-2174-44b5-9c94-410d0da236d6-kube-api-access-8sv6k\") pod \"limitador-limitador-859c4d96d6-lnqpp\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:06.846417 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:06.846387    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:07.115671 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.115588    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:07.119800 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:01:07.119774    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5868856_2174_44b5_9c94_410d0da236d6.slice/crio-dd21fb0cd0cbee274a73e683e61204284f585252b716e58cec884b506ae93acd WatchSource:0}: Error finding container dd21fb0cd0cbee274a73e683e61204284f585252b716e58cec884b506ae93acd: Status 404 returned error can't find the container with id dd21fb0cd0cbee274a73e683e61204284f585252b716e58cec884b506ae93acd
Sep 04 22:01:07.300148 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.300115    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp" event={"ID":"b5868856-2174-44b5-9c94-410d0da236d6","Type":"ContainerStarted","Data":"dd21fb0cd0cbee274a73e683e61204284f585252b716e58cec884b506ae93acd"}
Sep 04 22:01:07.306800 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.306776    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-fg6z5"]
Sep 04 22:01:07.309946 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.309932    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:07.312400 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.312380    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d9fqh\""
Sep 04 22:01:07.316163 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.316140    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fg6z5"]
Sep 04 22:01:07.367029 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.366957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmnfx\" (UniqueName: \"kubernetes.io/projected/7cf8e6be-7a44-46a6-8409-f8b8a92e5361-kube-api-access-gmnfx\") pod \"authorino-b54df94b4-fg6z5\" (UID: \"7cf8e6be-7a44-46a6-8409-f8b8a92e5361\") " pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:07.467706 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.467677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmnfx\" (UniqueName: \"kubernetes.io/projected/7cf8e6be-7a44-46a6-8409-f8b8a92e5361-kube-api-access-gmnfx\") pod \"authorino-b54df94b4-fg6z5\" (UID: \"7cf8e6be-7a44-46a6-8409-f8b8a92e5361\") " pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:07.476472 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.476448    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmnfx\" (UniqueName: \"kubernetes.io/projected/7cf8e6be-7a44-46a6-8409-f8b8a92e5361-kube-api-access-gmnfx\") pod \"authorino-b54df94b4-fg6z5\" (UID: \"7cf8e6be-7a44-46a6-8409-f8b8a92e5361\") " pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:07.619485 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.619404    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:07.889994 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:07.889923    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fg6z5"]
Sep 04 22:01:08.305909 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:08.305827    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fg6z5" event={"ID":"7cf8e6be-7a44-46a6-8409-f8b8a92e5361","Type":"ContainerStarted","Data":"0d96e3be14a4e759502f540a2e02e75cf054c76a0c729e617c51fc5e3e0869e2"}
Sep 04 22:01:10.193163 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:10.193130    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fg6z5"]
Sep 04 22:01:11.322220 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.322183    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp" event={"ID":"b5868856-2174-44b5-9c94-410d0da236d6","Type":"ContainerStarted","Data":"8c38408e69da3449b209e79c545b6010050038f7060fe6f7b8745b5ec19896de"}
Sep 04 22:01:11.322698 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.322309    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:11.323811 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.323793    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fg6z5" event={"ID":"7cf8e6be-7a44-46a6-8409-f8b8a92e5361","Type":"ContainerStarted","Data":"564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e"}
Sep 04 22:01:11.323901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.323829    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-fg6z5" podUID="7cf8e6be-7a44-46a6-8409-f8b8a92e5361" containerName="authorino" containerID="cri-o://564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e" gracePeriod=30
Sep 04 22:01:11.339000 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.338958    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp" podStartSLOduration=1.9733258660000002 podStartE2EDuration="5.338944402s" totalImagesPullingTime="3.365618536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:11.337964846 +0000 UTC m=+446.422349691" watchObservedRunningTime="2026-09-04 22:01:11.338944402 +0000 UTC m=+446.423329246"
Sep 04 22:01:11.354205 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.354163    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-fg6z5" podStartSLOduration=1.76074509 podStartE2EDuration="4.354148762s" totalImagesPullingTime="2.593403672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:11.352649266 +0000 UTC m=+446.437034110" watchObservedRunningTime="2026-09-04 22:01:11.354148762 +0000 UTC m=+446.438533607"
Sep 04 22:01:11.574991 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.574971    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:11.603690 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.603662    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmnfx\" (UniqueName: \"kubernetes.io/projected/7cf8e6be-7a44-46a6-8409-f8b8a92e5361-kube-api-access-gmnfx\") pod \"7cf8e6be-7a44-46a6-8409-f8b8a92e5361\" (UID: \"7cf8e6be-7a44-46a6-8409-f8b8a92e5361\") "
Sep 04 22:01:11.605970 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.605940    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf8e6be-7a44-46a6-8409-f8b8a92e5361-kube-api-access-gmnfx" (OuterVolumeSpecName: "kube-api-access-gmnfx") pod "7cf8e6be-7a44-46a6-8409-f8b8a92e5361" (UID: "7cf8e6be-7a44-46a6-8409-f8b8a92e5361"). InnerVolumeSpecName "kube-api-access-gmnfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:01:11.704254 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:11.704215    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmnfx\" (UniqueName: \"kubernetes.io/projected/7cf8e6be-7a44-46a6-8409-f8b8a92e5361-kube-api-access-gmnfx\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:01:12.329138 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.329103    2565 generic.go:356] "Generic (PLEG): container finished" podID="7cf8e6be-7a44-46a6-8409-f8b8a92e5361" containerID="564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e" exitCode=0
Sep 04 22:01:12.329612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.329144    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fg6z5"
Sep 04 22:01:12.329612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.329190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fg6z5" event={"ID":"7cf8e6be-7a44-46a6-8409-f8b8a92e5361","Type":"ContainerDied","Data":"564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e"}
Sep 04 22:01:12.329612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.329233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fg6z5" event={"ID":"7cf8e6be-7a44-46a6-8409-f8b8a92e5361","Type":"ContainerDied","Data":"0d96e3be14a4e759502f540a2e02e75cf054c76a0c729e617c51fc5e3e0869e2"}
Sep 04 22:01:12.329612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.329252    2565 scope.go:117] "RemoveContainer" containerID="564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e"
Sep 04 22:01:12.337905 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.337889    2565 scope.go:117] "RemoveContainer" containerID="564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e"
Sep 04 22:01:12.338149 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:01:12.338132    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e\": container with ID starting with 564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e not found: ID does not exist" containerID="564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e"
Sep 04 22:01:12.338200 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.338159    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e"} err="failed to get container status \"564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e\": rpc error: code = NotFound desc = could not find container \"564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e\": container with ID starting with 564bcd1ed4621df5f799772e6bb86c2f2ca7d4df81beea32342e2f721805be6e not found: ID does not exist"
Sep 04 22:01:12.351695 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.351673    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fg6z5"]
Sep 04 22:01:12.356130 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:12.356110    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fg6z5"]
Sep 04 22:01:13.389059 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:13.389031    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf8e6be-7a44-46a6-8409-f8b8a92e5361" path="/var/lib/kubelet/pods/7cf8e6be-7a44-46a6-8409-f8b8a92e5361/volumes"
Sep 04 22:01:22.330421 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:22.330388    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:28.519515 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:28.519479    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:28.519935 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:28.519674    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp" podUID="b5868856-2174-44b5-9c94-410d0da236d6" containerName="limitador" containerID="cri-o://8c38408e69da3449b209e79c545b6010050038f7060fe6f7b8745b5ec19896de" gracePeriod=30
Sep 04 22:01:29.404040 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.404006    2565 generic.go:356] "Generic (PLEG): container finished" podID="b5868856-2174-44b5-9c94-410d0da236d6" containerID="8c38408e69da3449b209e79c545b6010050038f7060fe6f7b8745b5ec19896de" exitCode=0
Sep 04 22:01:29.404182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.404074    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp" event={"ID":"b5868856-2174-44b5-9c94-410d0da236d6","Type":"ContainerDied","Data":"8c38408e69da3449b209e79c545b6010050038f7060fe6f7b8745b5ec19896de"}
Sep 04 22:01:29.462744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.462716    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:29.555999 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.555970    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5868856-2174-44b5-9c94-410d0da236d6-config-file\") pod \"b5868856-2174-44b5-9c94-410d0da236d6\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") "
Sep 04 22:01:29.556364 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.556036    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sv6k\" (UniqueName: \"kubernetes.io/projected/b5868856-2174-44b5-9c94-410d0da236d6-kube-api-access-8sv6k\") pod \"b5868856-2174-44b5-9c94-410d0da236d6\" (UID: \"b5868856-2174-44b5-9c94-410d0da236d6\") "
Sep 04 22:01:29.556364 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.556294    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5868856-2174-44b5-9c94-410d0da236d6-config-file" (OuterVolumeSpecName: "config-file") pod "b5868856-2174-44b5-9c94-410d0da236d6" (UID: "b5868856-2174-44b5-9c94-410d0da236d6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:01:29.558256 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.558237    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5868856-2174-44b5-9c94-410d0da236d6-kube-api-access-8sv6k" (OuterVolumeSpecName: "kube-api-access-8sv6k") pod "b5868856-2174-44b5-9c94-410d0da236d6" (UID: "b5868856-2174-44b5-9c94-410d0da236d6"). InnerVolumeSpecName "kube-api-access-8sv6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:01:29.657443 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.657364    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sv6k\" (UniqueName: \"kubernetes.io/projected/b5868856-2174-44b5-9c94-410d0da236d6-kube-api-access-8sv6k\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:01:29.657443 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:29.657389    2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b5868856-2174-44b5-9c94-410d0da236d6-config-file\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:01:30.409901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:30.409865    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp" event={"ID":"b5868856-2174-44b5-9c94-410d0da236d6","Type":"ContainerDied","Data":"dd21fb0cd0cbee274a73e683e61204284f585252b716e58cec884b506ae93acd"}
Sep 04 22:01:30.409901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:30.409896    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"
Sep 04 22:01:30.410111 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:30.409923    2565 scope.go:117] "RemoveContainer" containerID="8c38408e69da3449b209e79c545b6010050038f7060fe6f7b8745b5ec19896de"
Sep 04 22:01:30.429189 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:30.429117    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:30.430780 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:30.430759    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-lnqpp"]
Sep 04 22:01:31.389520 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:31.389489    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5868856-2174-44b5-9c94-410d0da236d6" path="/var/lib/kubelet/pods/b5868856-2174-44b5-9c94-410d0da236d6/volumes"
Sep 04 22:01:33.293323 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293287    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"]
Sep 04 22:01:33.293792 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293769    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf8e6be-7a44-46a6-8409-f8b8a92e5361" containerName="authorino"
Sep 04 22:01:33.293792 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293786    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf8e6be-7a44-46a6-8409-f8b8a92e5361" containerName="authorino"
Sep 04 22:01:33.293866 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293823    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5868856-2174-44b5-9c94-410d0da236d6" containerName="limitador"
Sep 04 22:01:33.293866 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293831    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5868856-2174-44b5-9c94-410d0da236d6" containerName="limitador"
Sep 04 22:01:33.293928 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293897    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf8e6be-7a44-46a6-8409-f8b8a92e5361" containerName="authorino"
Sep 04 22:01:33.293928 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.293912    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5868856-2174-44b5-9c94-410d0da236d6" containerName="limitador"
Sep 04 22:01:33.298998 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.298976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.302180 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.302157    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 04 22:01:33.302447 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.302424    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8bzmd\""
Sep 04 22:01:33.302659 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.302643    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:01:33.303291 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.303267    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 04 22:01:33.308137 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.308113    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"]
Sep 04 22:01:33.388797 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.388757    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8131eac8-04a1-466b-a932-320b8c9e6c4e-data\") pod \"postgres-69fb4594cf-qv7n9\" (UID: \"8131eac8-04a1-466b-a932-320b8c9e6c4e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.388958 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.388813    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsgm7\" (UniqueName: \"kubernetes.io/projected/8131eac8-04a1-466b-a932-320b8c9e6c4e-kube-api-access-fsgm7\") pod \"postgres-69fb4594cf-qv7n9\" (UID: \"8131eac8-04a1-466b-a932-320b8c9e6c4e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.489522 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.489482    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8131eac8-04a1-466b-a932-320b8c9e6c4e-data\") pod \"postgres-69fb4594cf-qv7n9\" (UID: \"8131eac8-04a1-466b-a932-320b8c9e6c4e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.489692 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.489529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsgm7\" (UniqueName: \"kubernetes.io/projected/8131eac8-04a1-466b-a932-320b8c9e6c4e-kube-api-access-fsgm7\") pod \"postgres-69fb4594cf-qv7n9\" (UID: \"8131eac8-04a1-466b-a932-320b8c9e6c4e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.489874 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.489850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8131eac8-04a1-466b-a932-320b8c9e6c4e-data\") pod \"postgres-69fb4594cf-qv7n9\" (UID: \"8131eac8-04a1-466b-a932-320b8c9e6c4e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.497780 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.497749    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsgm7\" (UniqueName: \"kubernetes.io/projected/8131eac8-04a1-466b-a932-320b8c9e6c4e-kube-api-access-fsgm7\") pod \"postgres-69fb4594cf-qv7n9\" (UID: \"8131eac8-04a1-466b-a932-320b8c9e6c4e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.610901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.610870    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:33.883489 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:33.883300    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"]
Sep 04 22:01:33.886061 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:01:33.886033    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8131eac8_04a1_466b_a932_320b8c9e6c4e.slice/crio-6b536d330240d15b79cd1135e4196df2d8becb406ae58af60c412712d9068cfd WatchSource:0}: Error finding container 6b536d330240d15b79cd1135e4196df2d8becb406ae58af60c412712d9068cfd: Status 404 returned error can't find the container with id 6b536d330240d15b79cd1135e4196df2d8becb406ae58af60c412712d9068cfd
Sep 04 22:01:34.429058 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:34.429022    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9" event={"ID":"8131eac8-04a1-466b-a932-320b8c9e6c4e","Type":"ContainerStarted","Data":"6b536d330240d15b79cd1135e4196df2d8becb406ae58af60c412712d9068cfd"}
Sep 04 22:01:39.457951 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:39.457918    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9" event={"ID":"8131eac8-04a1-466b-a932-320b8c9e6c4e","Type":"ContainerStarted","Data":"d632b820c99b83a59726a175489ae2dbd9f5f1b4e184f64600c5db915c098bd2"}
Sep 04 22:01:39.458401 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:39.458036    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:39.475539 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:39.475478    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9" podStartSLOduration=1.343176582 podStartE2EDuration="6.475458098s" totalImagesPullingTime="5.132281516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:39.473132187 +0000 UTC m=+474.557517032" watchObservedRunningTime="2026-09-04 22:01:39.475458098 +0000 UTC m=+474.559842942"
Sep 04 22:01:45.494107 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:45.494069    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qv7n9"
Sep 04 22:01:46.087347 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.087314    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-9gkrl"]
Sep 04 22:01:46.103138 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.103091    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-9gkrl"]
Sep 04 22:01:46.103289 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.103146    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:46.105613 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.105586    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d9fqh\""
Sep 04 22:01:46.105613 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.105604    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 22:01:46.105802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.105604    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 22:01:46.204546 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.204519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwqrh\" (UniqueName: \"kubernetes.io/projected/f676ca51-4cb4-4b13-8443-566aa2240405-kube-api-access-cwqrh\") pod \"authorino-5dc7ccb5f-9gkrl\" (UID: \"f676ca51-4cb4-4b13-8443-566aa2240405\") " pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:46.305458 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.305424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwqrh\" (UniqueName: \"kubernetes.io/projected/f676ca51-4cb4-4b13-8443-566aa2240405-kube-api-access-cwqrh\") pod \"authorino-5dc7ccb5f-9gkrl\" (UID: \"f676ca51-4cb4-4b13-8443-566aa2240405\") " pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:46.313815 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.313789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwqrh\" (UniqueName: \"kubernetes.io/projected/f676ca51-4cb4-4b13-8443-566aa2240405-kube-api-access-cwqrh\") pod \"authorino-5dc7ccb5f-9gkrl\" (UID: \"f676ca51-4cb4-4b13-8443-566aa2240405\") " pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:46.332780 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.332745    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-9gkrl"]
Sep 04 22:01:46.332967 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.332955    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:46.596769 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:46.596724    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-9gkrl"]
Sep 04 22:01:46.598830 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:01:46.598801    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf676ca51_4cb4_4b13_8443_566aa2240405.slice/crio-39b61bf6339fe2a61823dfac34a73a0e9048795d506cdd788155d25738ab70ac WatchSource:0}: Error finding container 39b61bf6339fe2a61823dfac34a73a0e9048795d506cdd788155d25738ab70ac: Status 404 returned error can't find the container with id 39b61bf6339fe2a61823dfac34a73a0e9048795d506cdd788155d25738ab70ac
Sep 04 22:01:47.490774 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.490693    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl" event={"ID":"f676ca51-4cb4-4b13-8443-566aa2240405","Type":"ContainerStarted","Data":"a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8"}
Sep 04 22:01:47.490774 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.490731    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl" event={"ID":"f676ca51-4cb4-4b13-8443-566aa2240405","Type":"ContainerStarted","Data":"39b61bf6339fe2a61823dfac34a73a0e9048795d506cdd788155d25738ab70ac"}
Sep 04 22:01:47.490774 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.490737    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl" podUID="f676ca51-4cb4-4b13-8443-566aa2240405" containerName="authorino" containerID="cri-o://a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8" gracePeriod=30
Sep 04 22:01:47.507434 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.507372    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl" podStartSLOduration=0.916619221 podStartE2EDuration="1.507325954s" totalImagesPullingTime="590.706733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:47.506965207 +0000 UTC m=+482.591350052" watchObservedRunningTime="2026-09-04 22:01:47.507325954 +0000 UTC m=+482.591710799"
Sep 04 22:01:47.730493 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.730471    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:47.925792 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.925763    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwqrh\" (UniqueName: \"kubernetes.io/projected/f676ca51-4cb4-4b13-8443-566aa2240405-kube-api-access-cwqrh\") pod \"f676ca51-4cb4-4b13-8443-566aa2240405\" (UID: \"f676ca51-4cb4-4b13-8443-566aa2240405\") "
Sep 04 22:01:47.927880 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:47.927857    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f676ca51-4cb4-4b13-8443-566aa2240405-kube-api-access-cwqrh" (OuterVolumeSpecName: "kube-api-access-cwqrh") pod "f676ca51-4cb4-4b13-8443-566aa2240405" (UID: "f676ca51-4cb4-4b13-8443-566aa2240405"). InnerVolumeSpecName "kube-api-access-cwqrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:01:48.027068 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.027040    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwqrh\" (UniqueName: \"kubernetes.io/projected/f676ca51-4cb4-4b13-8443-566aa2240405-kube-api-access-cwqrh\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:01:48.495984 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.495948    2565 generic.go:356] "Generic (PLEG): container finished" podID="f676ca51-4cb4-4b13-8443-566aa2240405" containerID="a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8" exitCode=0
Sep 04 22:01:48.496129 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.495995    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl"
Sep 04 22:01:48.496129 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.496035    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl" event={"ID":"f676ca51-4cb4-4b13-8443-566aa2240405","Type":"ContainerDied","Data":"a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8"}
Sep 04 22:01:48.496129 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.496078    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-9gkrl" event={"ID":"f676ca51-4cb4-4b13-8443-566aa2240405","Type":"ContainerDied","Data":"39b61bf6339fe2a61823dfac34a73a0e9048795d506cdd788155d25738ab70ac"}
Sep 04 22:01:48.496129 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.496099    2565 scope.go:117] "RemoveContainer" containerID="a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8"
Sep 04 22:01:48.504680 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.504664    2565 scope.go:117] "RemoveContainer" containerID="a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8"
Sep 04 22:01:48.504949 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:01:48.504932    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8\": container with ID starting with a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8 not found: ID does not exist" containerID="a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8"
Sep 04 22:01:48.504995 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.504958    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8"} err="failed to get container status \"a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8\": rpc error: code = NotFound desc = could not find container \"a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8\": container with ID starting with a4c9ba80ab0c4f328a78763646d46e2d1a632e830629922feb32595d178232a8 not found: ID does not exist"
Sep 04 22:01:48.517315 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.517293    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-9gkrl"]
Sep 04 22:01:48.518621 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:48.518604    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-9gkrl"]
Sep 04 22:01:49.389253 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:49.389218    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f676ca51-4cb4-4b13-8443-566aa2240405" path="/var/lib/kubelet/pods/f676ca51-4cb4-4b13-8443-566aa2240405/volumes"
Sep 04 22:01:53.185950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.185916    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66856bcd5c-9njhx"]
Sep 04 22:01:53.186283 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.186233    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f676ca51-4cb4-4b13-8443-566aa2240405" containerName="authorino"
Sep 04 22:01:53.186283 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.186244    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f676ca51-4cb4-4b13-8443-566aa2240405" containerName="authorino"
Sep 04 22:01:53.186396 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.186302    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f676ca51-4cb4-4b13-8443-566aa2240405" containerName="authorino"
Sep 04 22:01:53.189978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.189961    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.192239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.192218    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 04 22:01:53.192239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.192237    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 04 22:01:53.192437 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.192238    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 04 22:01:53.192512 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.192500    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nwkwk\""
Sep 04 22:01:53.197788 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.197764    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66856bcd5c-9njhx"]
Sep 04 22:01:53.269148 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.269119    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdd4\" (UniqueName: \"kubernetes.io/projected/bd5e26b5-0205-451d-b7c5-473407e38ab6-kube-api-access-jpdd4\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.269326 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.269182    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-webhook-cert\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.269326 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.269259    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.370268 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.370234    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.370462 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.370303    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdd4\" (UniqueName: \"kubernetes.io/projected/bd5e26b5-0205-451d-b7c5-473407e38ab6-kube-api-access-jpdd4\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.370462 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.370378    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-webhook-cert\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.370462 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:01:53.370401    2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 04 22:01:53.370584 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:01:53.370471    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs podName:bd5e26b5-0205-451d-b7c5-473407e38ab6 nodeName:}" failed. No retries permitted until 2026-09-04 22:01:53.870448816 +0000 UTC m=+488.954833655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs") pod "maas-controller-66856bcd5c-9njhx" (UID: "bd5e26b5-0205-451d-b7c5-473407e38ab6") : secret "maas-controller-metrics-tls" not found
Sep 04 22:01:53.372859 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.372838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-webhook-cert\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.376027 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.376002    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78b457d45d-jvwc8"]
Sep 04 22:01:53.378531 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.378514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.380937 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.380912    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdd4\" (UniqueName: \"kubernetes.io/projected/bd5e26b5-0205-451d-b7c5-473407e38ab6-kube-api-access-jpdd4\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.391276 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.391247    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78b457d45d-jvwc8"]
Sep 04 22:01:53.471013 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.470908    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-metrics-certs\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.471013 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.470955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-webhook-cert\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.471240 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.471128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkqqm\" (UniqueName: \"kubernetes.io/projected/ae07492c-0540-4230-a3e8-7b8cb681ca65-kube-api-access-hkqqm\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.571716 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.571683    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkqqm\" (UniqueName: \"kubernetes.io/projected/ae07492c-0540-4230-a3e8-7b8cb681ca65-kube-api-access-hkqqm\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.571886 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.571723    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-metrics-certs\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.571886 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.571742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-webhook-cert\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.574277 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.574246    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-metrics-certs\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.574389 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.574305    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-webhook-cert\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.580213 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.580192    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkqqm\" (UniqueName: \"kubernetes.io/projected/ae07492c-0540-4230-a3e8-7b8cb681ca65-kube-api-access-hkqqm\") pod \"maas-controller-78b457d45d-jvwc8\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") " pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.696990 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.696954    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:53.875328 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.875287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.877765 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.877745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs\") pod \"maas-controller-66856bcd5c-9njhx\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") " pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:53.968082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:53.968060    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78b457d45d-jvwc8"]
Sep 04 22:01:53.970723 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:01:53.970702    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae07492c_0540_4230_a3e8_7b8cb681ca65.slice/crio-61ea6a495491c605d62112bf9e7fa75a1e70eaa7af8eeb1b12ee432acfc1bea6 WatchSource:0}: Error finding container 61ea6a495491c605d62112bf9e7fa75a1e70eaa7af8eeb1b12ee432acfc1bea6: Status 404 returned error can't find the container with id 61ea6a495491c605d62112bf9e7fa75a1e70eaa7af8eeb1b12ee432acfc1bea6
Sep 04 22:01:54.100249 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:54.100215    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:54.370953 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:54.370871    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66856bcd5c-9njhx"]
Sep 04 22:01:54.373636 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:01:54.373610    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5e26b5_0205_451d_b7c5_473407e38ab6.slice/crio-115ee89b56313197cdb7fbcc0ff07c3e84ece83a6ee8aa307a8128c02a2fd680 WatchSource:0}: Error finding container 115ee89b56313197cdb7fbcc0ff07c3e84ece83a6ee8aa307a8128c02a2fd680: Status 404 returned error can't find the container with id 115ee89b56313197cdb7fbcc0ff07c3e84ece83a6ee8aa307a8128c02a2fd680
Sep 04 22:01:54.523390 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:54.523355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-jvwc8" event={"ID":"ae07492c-0540-4230-a3e8-7b8cb681ca65","Type":"ContainerStarted","Data":"61ea6a495491c605d62112bf9e7fa75a1e70eaa7af8eeb1b12ee432acfc1bea6"}
Sep 04 22:01:54.524512 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:54.524490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66856bcd5c-9njhx" event={"ID":"bd5e26b5-0205-451d-b7c5-473407e38ab6","Type":"ContainerStarted","Data":"115ee89b56313197cdb7fbcc0ff07c3e84ece83a6ee8aa307a8128c02a2fd680"}
Sep 04 22:01:57.541056 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:57.541021    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66856bcd5c-9njhx" event={"ID":"bd5e26b5-0205-451d-b7c5-473407e38ab6","Type":"ContainerStarted","Data":"472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1"}
Sep 04 22:01:57.541516 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:57.541190    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:01:57.542595 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:57.542572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-jvwc8" event={"ID":"ae07492c-0540-4230-a3e8-7b8cb681ca65","Type":"ContainerStarted","Data":"a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564"}
Sep 04 22:01:57.542705 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:57.542691    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:01:57.561439 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:57.561394    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66856bcd5c-9njhx" podStartSLOduration=2.347473529 podStartE2EDuration="4.561379735s" totalImagesPullingTime="2.213906206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:57.560145754 +0000 UTC m=+492.644530612" watchObservedRunningTime="2026-09-04 22:01:57.561379735 +0000 UTC m=+492.645764579"
Sep 04 22:01:57.577377 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:01:57.577314    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78b457d45d-jvwc8" podStartSLOduration=1.969795978 podStartE2EDuration="4.577296741s" totalImagesPullingTime="2.607500763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:57.577228128 +0000 UTC m=+492.661612994" watchObservedRunningTime="2026-09-04 22:01:57.577296741 +0000 UTC m=+492.661681586"
Sep 04 22:02:08.549073 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.549044    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:02:08.549448 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.549110    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:02:08.611773 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.611744    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66856bcd5c-9njhx"]
Sep 04 22:02:08.611936 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.611913    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66856bcd5c-9njhx" podUID="bd5e26b5-0205-451d-b7c5-473407e38ab6" containerName="manager" containerID="cri-o://472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1" gracePeriod=30
Sep 04 22:02:08.845133 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.845107    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:02:08.912250 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.912210    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs\") pod \"bd5e26b5-0205-451d-b7c5-473407e38ab6\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") "
Sep 04 22:02:08.912463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.912306    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpdd4\" (UniqueName: \"kubernetes.io/projected/bd5e26b5-0205-451d-b7c5-473407e38ab6-kube-api-access-jpdd4\") pod \"bd5e26b5-0205-451d-b7c5-473407e38ab6\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") "
Sep 04 22:02:08.912463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.912381    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-webhook-cert\") pod \"bd5e26b5-0205-451d-b7c5-473407e38ab6\" (UID: \"bd5e26b5-0205-451d-b7c5-473407e38ab6\") "
Sep 04 22:02:08.914597 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.914562    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bd5e26b5-0205-451d-b7c5-473407e38ab6" (UID: "bd5e26b5-0205-451d-b7c5-473407e38ab6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:02:08.914597 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.914584    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd5e26b5-0205-451d-b7c5-473407e38ab6-kube-api-access-jpdd4" (OuterVolumeSpecName: "kube-api-access-jpdd4") pod "bd5e26b5-0205-451d-b7c5-473407e38ab6" (UID: "bd5e26b5-0205-451d-b7c5-473407e38ab6"). InnerVolumeSpecName "kube-api-access-jpdd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:02:08.914764 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:08.914619    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bd5e26b5-0205-451d-b7c5-473407e38ab6" (UID: "bd5e26b5-0205-451d-b7c5-473407e38ab6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:02:09.013392 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.013328    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:02:09.013392 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.013387    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpdd4\" (UniqueName: \"kubernetes.io/projected/bd5e26b5-0205-451d-b7c5-473407e38ab6-kube-api-access-jpdd4\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:02:09.013392 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.013396    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd5e26b5-0205-451d-b7c5-473407e38ab6-webhook-cert\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:02:09.596856 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.596818    2565 generic.go:356] "Generic (PLEG): container finished" podID="bd5e26b5-0205-451d-b7c5-473407e38ab6" containerID="472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1" exitCode=0
Sep 04 22:02:09.597293 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.596867    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66856bcd5c-9njhx"
Sep 04 22:02:09.597293 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.596904    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66856bcd5c-9njhx" event={"ID":"bd5e26b5-0205-451d-b7c5-473407e38ab6","Type":"ContainerDied","Data":"472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1"}
Sep 04 22:02:09.597293 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.596967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66856bcd5c-9njhx" event={"ID":"bd5e26b5-0205-451d-b7c5-473407e38ab6","Type":"ContainerDied","Data":"115ee89b56313197cdb7fbcc0ff07c3e84ece83a6ee8aa307a8128c02a2fd680"}
Sep 04 22:02:09.597293 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.597006    2565 scope.go:117] "RemoveContainer" containerID="472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1"
Sep 04 22:02:09.605257 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.605207    2565 scope.go:117] "RemoveContainer" containerID="472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1"
Sep 04 22:02:09.605512 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:02:09.605492    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1\": container with ID starting with 472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1 not found: ID does not exist" containerID="472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1"
Sep 04 22:02:09.605580 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.605519    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1"} err="failed to get container status \"472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1\": rpc error: code = NotFound desc = could not find container \"472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1\": container with ID starting with 472897f2b84434d4b7144d3dd94b52ea1e4056ca392e2ef8b7f581d7a2c55ba1 not found: ID does not exist"
Sep 04 22:02:09.611831 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.611810    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66856bcd5c-9njhx"]
Sep 04 22:02:09.615545 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:09.615525    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66856bcd5c-9njhx"]
Sep 04 22:02:11.389637 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:11.389606    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd5e26b5-0205-451d-b7c5-473407e38ab6" path="/var/lib/kubelet/pods/bd5e26b5-0205-451d-b7c5-473407e38ab6/volumes"
Sep 04 22:02:32.004034 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.003995    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"]
Sep 04 22:02:32.004408 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.004377    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd5e26b5-0205-451d-b7c5-473407e38ab6" containerName="manager"
Sep 04 22:02:32.004408 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.004392    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd5e26b5-0205-451d-b7c5-473407e38ab6" containerName="manager"
Sep 04 22:02:32.004483 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.004444    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd5e26b5-0205-451d-b7c5-473407e38ab6" containerName="manager"
Sep 04 22:02:32.016222 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.016204    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.018599 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.018565    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 04 22:02:32.018815 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.018751    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 04 22:02:32.019069 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.018837    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2qzmr\""
Sep 04 22:02:32.019756 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.019734    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"]
Sep 04 22:02:32.020135 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.020119    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8pf9\" (UniqueName: \"kubernetes.io/projected/c960dee6-9338-4b91-b9d1-1351e3f0bb07-kube-api-access-p8pf9\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.020191 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.020152    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-maas-api-tls\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.020191 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.020172    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c960dee6-9338-4b91-b9d1-1351e3f0bb07-openshift-service-ca\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.020278 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.020242    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.121385 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.121315    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8pf9\" (UniqueName: \"kubernetes.io/projected/c960dee6-9338-4b91-b9d1-1351e3f0bb07-kube-api-access-p8pf9\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.121559 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.121407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-maas-api-tls\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.121559 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.121432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c960dee6-9338-4b91-b9d1-1351e3f0bb07-openshift-service-ca\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.121559 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.121459    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.121657 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:02:32.121595    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 04 22:02:32.121703 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:02:32.121690    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs podName:c960dee6-9338-4b91-b9d1-1351e3f0bb07 nodeName:}" failed. No retries permitted until 2026-09-04 22:02:32.621668964 +0000 UTC m=+527.706053790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs") pod "maas-api-65db659b7c-mxnkt" (UID: "c960dee6-9338-4b91-b9d1-1351e3f0bb07") : secret "maas-api-metrics-tls" not found
Sep 04 22:02:32.122143 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.122126    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c960dee6-9338-4b91-b9d1-1351e3f0bb07-openshift-service-ca\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.124043 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.124022    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-maas-api-tls\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.130718 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.130697    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8pf9\" (UniqueName: \"kubernetes.io/projected/c960dee6-9338-4b91-b9d1-1351e3f0bb07-kube-api-access-p8pf9\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.489143 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.489117    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"]
Sep 04 22:02:32.492669 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.492650    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.494907 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.494886    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bpvpg\""
Sep 04 22:02:32.495011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.494957    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 04 22:02:32.503131 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.503104    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"]
Sep 04 22:02:32.524112 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.524080    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f14f6-59e1-4c21-8441-a8bcffcb193d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-s8xqq\" (UID: \"441f14f6-59e1-4c21-8441-a8bcffcb193d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.524232 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.524138    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmwh\" (UniqueName: \"kubernetes.io/projected/441f14f6-59e1-4c21-8441-a8bcffcb193d-kube-api-access-sxmwh\") pod \"payload-pre-processing-59fbc47bf8-s8xqq\" (UID: \"441f14f6-59e1-4c21-8441-a8bcffcb193d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.625590 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.625561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:32.625759 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.625619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f14f6-59e1-4c21-8441-a8bcffcb193d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-s8xqq\" (UID: \"441f14f6-59e1-4c21-8441-a8bcffcb193d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.625759 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.625659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmwh\" (UniqueName: \"kubernetes.io/projected/441f14f6-59e1-4c21-8441-a8bcffcb193d-kube-api-access-sxmwh\") pod \"payload-pre-processing-59fbc47bf8-s8xqq\" (UID: \"441f14f6-59e1-4c21-8441-a8bcffcb193d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.625759 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:02:32.625725    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 04 22:02:32.625922 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:02:32.625813    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs podName:c960dee6-9338-4b91-b9d1-1351e3f0bb07 nodeName:}" failed. No retries permitted until 2026-09-04 22:02:33.625790815 +0000 UTC m=+528.710175641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs") pod "maas-api-65db659b7c-mxnkt" (UID: "c960dee6-9338-4b91-b9d1-1351e3f0bb07") : secret "maas-api-metrics-tls" not found
Sep 04 22:02:32.626373 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.626333    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f14f6-59e1-4c21-8441-a8bcffcb193d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-s8xqq\" (UID: \"441f14f6-59e1-4c21-8441-a8bcffcb193d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.635747 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.635724    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmwh\" (UniqueName: \"kubernetes.io/projected/441f14f6-59e1-4c21-8441-a8bcffcb193d-kube-api-access-sxmwh\") pod \"payload-pre-processing-59fbc47bf8-s8xqq\" (UID: \"441f14f6-59e1-4c21-8441-a8bcffcb193d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:32.804084 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:32.803996    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:33.073670 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:33.073600    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"]
Sep 04 22:02:33.076830 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:02:33.076803    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441f14f6_59e1_4c21_8441_a8bcffcb193d.slice/crio-703caee5cd5db5cb5b4c2273e4c5b86d7de277c33ba062eeaeeec983ebacba5b WatchSource:0}: Error finding container 703caee5cd5db5cb5b4c2273e4c5b86d7de277c33ba062eeaeeec983ebacba5b: Status 404 returned error can't find the container with id 703caee5cd5db5cb5b4c2273e4c5b86d7de277c33ba062eeaeeec983ebacba5b
Sep 04 22:02:33.634635 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:33.634580    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:33.637639 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:33.637588    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") pod \"maas-api-65db659b7c-mxnkt\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") " pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:33.707862 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:33.707780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq" event={"ID":"441f14f6-59e1-4c21-8441-a8bcffcb193d","Type":"ContainerStarted","Data":"703caee5cd5db5cb5b4c2273e4c5b86d7de277c33ba062eeaeeec983ebacba5b"}
Sep 04 22:02:33.827542 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:33.827495    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:34.107719 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:34.107685    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"]
Sep 04 22:02:34.111670 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:02:34.111642    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc960dee6_9338_4b91_b9d1_1351e3f0bb07.slice/crio-9079286e6fb39478cd813e8074757326848964555a4fa9fd062452e27dc1960c WatchSource:0}: Error finding container 9079286e6fb39478cd813e8074757326848964555a4fa9fd062452e27dc1960c: Status 404 returned error can't find the container with id 9079286e6fb39478cd813e8074757326848964555a4fa9fd062452e27dc1960c
Sep 04 22:02:34.715236 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:34.715197    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" event={"ID":"c960dee6-9338-4b91-b9d1-1351e3f0bb07","Type":"ContainerStarted","Data":"9079286e6fb39478cd813e8074757326848964555a4fa9fd062452e27dc1960c"}
Sep 04 22:02:35.721586 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:35.721497    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq" event={"ID":"441f14f6-59e1-4c21-8441-a8bcffcb193d","Type":"ContainerStarted","Data":"4f08517007ca5a65258488b002df18f3d89bdbd261f4d132209c9f320ef167a4"}
Sep 04 22:02:35.721586 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:35.721583    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:02:35.741003 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:35.740960    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq" podStartSLOduration=1.423435164 podStartE2EDuration="3.740942087s" totalImagesPullingTime="2.317506923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:02:35.739274802 +0000 UTC m=+530.823659649" watchObservedRunningTime="2026-09-04 22:02:35.740942087 +0000 UTC m=+530.825326933"
Sep 04 22:02:36.730426 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:36.730296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" event={"ID":"c960dee6-9338-4b91-b9d1-1351e3f0bb07","Type":"ContainerStarted","Data":"2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c"}
Sep 04 22:02:36.750624 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:36.750575    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" podStartSLOduration=3.517513091 podStartE2EDuration="5.750559594s" totalImagesPullingTime="2.233046503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:02:36.74950873 +0000 UTC m=+531.833893576" watchObservedRunningTime="2026-09-04 22:02:36.750559594 +0000 UTC m=+531.834944463"
Sep 04 22:02:37.735325 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:37.735289    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:43.745687 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:43.745659    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:02:46.732522 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:02:46.732490    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xqq"
Sep 04 22:03:07.441767 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.441726    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"]
Sep 04 22:03:07.446669 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.446647    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.449750 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.449732    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 04 22:03:07.449939 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.449926    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 04 22:03:07.450036 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.449945    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6vlzh\""
Sep 04 22:03:07.450036 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.449965    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 04 22:03:07.456476 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.456454    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"]
Sep 04 22:03:07.521369 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.521321    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.521524 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.521430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.521524 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.521499    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.521524 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.521517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.521690 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.521591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.521834 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.521802    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmff7\" (UniqueName: \"kubernetes.io/projected/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-kube-api-access-dmff7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623009 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.622977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmff7\" (UniqueName: \"kubernetes.io/projected/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-kube-api-access-dmff7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623175 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623037    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623175 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623069    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623175 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623092    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623175 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623175 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623443 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623544 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.623603 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.623541    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.625497 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.625467    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.625725 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.625709    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.630770 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.630754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmff7\" (UniqueName: \"kubernetes.io/projected/9f0826af-7ddb-41c7-a7a3-bf3cd0271119-kube-api-access-dmff7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p\" (UID: \"9f0826af-7ddb-41c7-a7a3-bf3cd0271119\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:07.758694 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:07.758610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:08.031661 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:08.031608    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"]
Sep 04 22:03:08.034681 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:03:08.034654    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0826af_7ddb_41c7_a7a3_bf3cd0271119.slice/crio-951686a28af675bf05b1713843cbebce381a7d9e348181a5ad2d4cae1a9f4660 WatchSource:0}: Error finding container 951686a28af675bf05b1713843cbebce381a7d9e348181a5ad2d4cae1a9f4660: Status 404 returned error can't find the container with id 951686a28af675bf05b1713843cbebce381a7d9e348181a5ad2d4cae1a9f4660
Sep 04 22:03:08.882705 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:08.882670    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p" event={"ID":"9f0826af-7ddb-41c7-a7a3-bf3cd0271119","Type":"ContainerStarted","Data":"951686a28af675bf05b1713843cbebce381a7d9e348181a5ad2d4cae1a9f4660"}
Sep 04 22:03:09.889055 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:09.889015    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p" event={"ID":"9f0826af-7ddb-41c7-a7a3-bf3cd0271119","Type":"ContainerStarted","Data":"e08e6901a5b8935505ae102dadc32cbd12d68b9af85a060057f9b30f63b127d0"}
Sep 04 22:03:09.914049 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:09.914003    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p" podStartSLOduration=1.713418362 podStartE2EDuration="2.913988569s" totalImagesPullingTime="1.200570207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:03:09.912646271 +0000 UTC m=+564.997031114" watchObservedRunningTime="2026-09-04 22:03:09.913988569 +0000 UTC m=+564.998373412"
Sep 04 22:03:17.759750 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:17.759713    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:17.760271 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:17.759791    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:17.773031 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:17.773010    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:03:17.935887 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:03:17.935857    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p"
Sep 04 22:05:12.136988 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.136952    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-786d4f99d5-kddnf"]
Sep 04 22:05:12.140274 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.140258    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.142569 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.142547    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 22:05:12.143212 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.143190    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 04 22:05:12.143314 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.143232    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 22:05:12.143314 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.143249    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d9fqh\""
Sep 04 22:05:12.148012 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.147993    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-786d4f99d5-kddnf"]
Sep 04 22:05:12.178070 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.178040    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9430c8a-e334-4f73-b71f-2ff7d5964b6b-tls-cert\") pod \"authorino-786d4f99d5-kddnf\" (UID: \"f9430c8a-e334-4f73-b71f-2ff7d5964b6b\") " pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.178202 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.178083    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w5ns\" (UniqueName: \"kubernetes.io/projected/f9430c8a-e334-4f73-b71f-2ff7d5964b6b-kube-api-access-2w5ns\") pod \"authorino-786d4f99d5-kddnf\" (UID: \"f9430c8a-e334-4f73-b71f-2ff7d5964b6b\") " pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.278973 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.278940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9430c8a-e334-4f73-b71f-2ff7d5964b6b-tls-cert\") pod \"authorino-786d4f99d5-kddnf\" (UID: \"f9430c8a-e334-4f73-b71f-2ff7d5964b6b\") " pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.279102 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.278982    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w5ns\" (UniqueName: \"kubernetes.io/projected/f9430c8a-e334-4f73-b71f-2ff7d5964b6b-kube-api-access-2w5ns\") pod \"authorino-786d4f99d5-kddnf\" (UID: \"f9430c8a-e334-4f73-b71f-2ff7d5964b6b\") " pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.281469 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.281441    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9430c8a-e334-4f73-b71f-2ff7d5964b6b-tls-cert\") pod \"authorino-786d4f99d5-kddnf\" (UID: \"f9430c8a-e334-4f73-b71f-2ff7d5964b6b\") " pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.287851 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.287823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w5ns\" (UniqueName: \"kubernetes.io/projected/f9430c8a-e334-4f73-b71f-2ff7d5964b6b-kube-api-access-2w5ns\") pod \"authorino-786d4f99d5-kddnf\" (UID: \"f9430c8a-e334-4f73-b71f-2ff7d5964b6b\") " pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.451160 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.451086    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-786d4f99d5-kddnf"
Sep 04 22:05:12.720730 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.720708    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-786d4f99d5-kddnf"]
Sep 04 22:05:12.722575 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:12.722548    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9430c8a_e334_4f73_b71f_2ff7d5964b6b.slice/crio-d9e8d5076f5c01f45a9b1b4fd799ad22e20ec6ffc3b64567f4729043007ce4ac WatchSource:0}: Error finding container d9e8d5076f5c01f45a9b1b4fd799ad22e20ec6ffc3b64567f4729043007ce4ac: Status 404 returned error can't find the container with id d9e8d5076f5c01f45a9b1b4fd799ad22e20ec6ffc3b64567f4729043007ce4ac
Sep 04 22:05:12.723952 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:12.723931    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:05:13.436201 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:13.436165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-786d4f99d5-kddnf" event={"ID":"f9430c8a-e334-4f73-b71f-2ff7d5964b6b","Type":"ContainerStarted","Data":"d9e8d5076f5c01f45a9b1b4fd799ad22e20ec6ffc3b64567f4729043007ce4ac"}
Sep 04 22:05:14.441999 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:14.441951    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-786d4f99d5-kddnf" event={"ID":"f9430c8a-e334-4f73-b71f-2ff7d5964b6b","Type":"ContainerStarted","Data":"d7607c2fd8247ae69122ae4de94ccc6b8a3d1fe75c7da5361333c6622a7f24d1"}
Sep 04 22:05:33.883237 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:33.883180    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-786d4f99d5-kddnf" podStartSLOduration=21.118028822 podStartE2EDuration="21.883162856s" totalImagesPullingTime="765.134034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:05:14.457617157 +0000 UTC m=+689.542002001" watchObservedRunningTime="2026-09-04 22:05:33.883162856 +0000 UTC m=+708.967547699"
Sep 04 22:05:33.883673 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:33.883449    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7827-140431937424560"]
Sep 04 22:05:33.886962 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:33.886941    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:33.890550 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:33.890524    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nhr85\""
Sep 04 22:05:33.892719 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:33.892690    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7827-140431937424560"]
Sep 04 22:05:33.973744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:33.973714    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7gsm\" (UniqueName: \"kubernetes.io/projected/669ef395-c349-43c4-b1b6-d13fcb86df78-kube-api-access-b7gsm\") pod \"test-curl-7827-140431937424560\" (UID: \"669ef395-c349-43c4-b1b6-d13fcb86df78\") " pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:34.074653 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:34.074605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7gsm\" (UniqueName: \"kubernetes.io/projected/669ef395-c349-43c4-b1b6-d13fcb86df78-kube-api-access-b7gsm\") pod \"test-curl-7827-140431937424560\" (UID: \"669ef395-c349-43c4-b1b6-d13fcb86df78\") " pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:34.085816 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:34.085782    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7gsm\" (UniqueName: \"kubernetes.io/projected/669ef395-c349-43c4-b1b6-d13fcb86df78-kube-api-access-b7gsm\") pod \"test-curl-7827-140431937424560\" (UID: \"669ef395-c349-43c4-b1b6-d13fcb86df78\") " pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:34.196639 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:34.196546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:34.467773 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:34.467744    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7827-140431937424560"]
Sep 04 22:05:34.470446 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:34.470403    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod669ef395_c349_43c4_b1b6_d13fcb86df78.slice/crio-8d06bb18469489efa5afa563545983f4742add8b4fc78f26084c6ad797229834 WatchSource:0}: Error finding container 8d06bb18469489efa5afa563545983f4742add8b4fc78f26084c6ad797229834: Status 404 returned error can't find the container with id 8d06bb18469489efa5afa563545983f4742add8b4fc78f26084c6ad797229834
Sep 04 22:05:34.535878 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:34.535847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7827-140431937424560" event={"ID":"669ef395-c349-43c4-b1b6-d13fcb86df78","Type":"ContainerStarted","Data":"8d06bb18469489efa5afa563545983f4742add8b4fc78f26084c6ad797229834"}
Sep 04 22:05:35.542390 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:35.542277    2565 generic.go:356] "Generic (PLEG): container finished" podID="669ef395-c349-43c4-b1b6-d13fcb86df78" containerID="610645f807f71f51581dc29cc1172f048521245b33876b374d212eae0c2d53c6" exitCode=0
Sep 04 22:05:35.542822 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:35.542395    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7827-140431937424560" event={"ID":"669ef395-c349-43c4-b1b6-d13fcb86df78","Type":"ContainerDied","Data":"610645f807f71f51581dc29cc1172f048521245b33876b374d212eae0c2d53c6"}
Sep 04 22:05:36.679857 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.679816    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:36.717276 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.717249    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7827-140431937424560_669ef395-c349-43c4-b1b6-d13fcb86df78/test-curl-7827-140431937424560/0.log"
Sep 04 22:05:36.737359 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.737311    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7827-140431937424560"]
Sep 04 22:05:36.741060 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.741026    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7827-140431937424560"]
Sep 04 22:05:36.801551 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.801480    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7gsm\" (UniqueName: \"kubernetes.io/projected/669ef395-c349-43c4-b1b6-d13fcb86df78-kube-api-access-b7gsm\") pod \"669ef395-c349-43c4-b1b6-d13fcb86df78\" (UID: \"669ef395-c349-43c4-b1b6-d13fcb86df78\") "
Sep 04 22:05:36.803631 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.803594    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/669ef395-c349-43c4-b1b6-d13fcb86df78-kube-api-access-b7gsm" (OuterVolumeSpecName: "kube-api-access-b7gsm") pod "669ef395-c349-43c4-b1b6-d13fcb86df78" (UID: "669ef395-c349-43c4-b1b6-d13fcb86df78"). InnerVolumeSpecName "kube-api-access-b7gsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:05:36.865908 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.865876    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7827-140431937424304"]
Sep 04 22:05:36.866245 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.866232    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="669ef395-c349-43c4-b1b6-d13fcb86df78" containerName="test-curl-7827-140431937424560"
Sep 04 22:05:36.866292 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.866246    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="669ef395-c349-43c4-b1b6-d13fcb86df78" containerName="test-curl-7827-140431937424560"
Sep 04 22:05:36.866333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.866318    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="669ef395-c349-43c4-b1b6-d13fcb86df78" containerName="test-curl-7827-140431937424560"
Sep 04 22:05:36.869155 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.869140    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:36.876417 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.876392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7827-140431937424304"]
Sep 04 22:05:36.902467 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:36.902442    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7gsm\" (UniqueName: \"kubernetes.io/projected/669ef395-c349-43c4-b1b6-d13fcb86df78-kube-api-access-b7gsm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:05:37.002871 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.002837    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgvrn\" (UniqueName: \"kubernetes.io/projected/944334f2-f969-4e09-9bc4-36d669923e2a-kube-api-access-mgvrn\") pod \"test-curl-7827-140431937424304\" (UID: \"944334f2-f969-4e09-9bc4-36d669923e2a\") " pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:37.104294 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.104265    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgvrn\" (UniqueName: \"kubernetes.io/projected/944334f2-f969-4e09-9bc4-36d669923e2a-kube-api-access-mgvrn\") pod \"test-curl-7827-140431937424304\" (UID: \"944334f2-f969-4e09-9bc4-36d669923e2a\") " pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:37.113268 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.113236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgvrn\" (UniqueName: \"kubernetes.io/projected/944334f2-f969-4e09-9bc4-36d669923e2a-kube-api-access-mgvrn\") pod \"test-curl-7827-140431937424304\" (UID: \"944334f2-f969-4e09-9bc4-36d669923e2a\") " pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:37.180181 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.180141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:37.390068 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.389987    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="669ef395-c349-43c4-b1b6-d13fcb86df78" path="/var/lib/kubelet/pods/669ef395-c349-43c4-b1b6-d13fcb86df78/volumes"
Sep 04 22:05:37.448405 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.448384    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7827-140431937424304"]
Sep 04 22:05:37.450437 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:37.450404    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod944334f2_f969_4e09_9bc4_36d669923e2a.slice/crio-3a562e1b54691d0bf85d730e480d21778160ed9e9f63fb96c05e3b622c868a1d WatchSource:0}: Error finding container 3a562e1b54691d0bf85d730e480d21778160ed9e9f63fb96c05e3b622c868a1d: Status 404 returned error can't find the container with id 3a562e1b54691d0bf85d730e480d21778160ed9e9f63fb96c05e3b622c868a1d
Sep 04 22:05:37.552760 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.552718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7827-140431937424304" event={"ID":"944334f2-f969-4e09-9bc4-36d669923e2a","Type":"ContainerStarted","Data":"3a562e1b54691d0bf85d730e480d21778160ed9e9f63fb96c05e3b622c868a1d"}
Sep 04 22:05:37.553985 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.553964    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424560"
Sep 04 22:05:37.554139 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:37.553972    2565 scope.go:117] "RemoveContainer" containerID="610645f807f71f51581dc29cc1172f048521245b33876b374d212eae0c2d53c6"
Sep 04 22:05:38.559734 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:38.559697    2565 generic.go:356] "Generic (PLEG): container finished" podID="944334f2-f969-4e09-9bc4-36d669923e2a" containerID="20315eee46db88f4e3bf025c5065a0be3b1dca77335bea6a509c8c425259a73d" exitCode=0
Sep 04 22:05:38.560165 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:38.559783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7827-140431937424304" event={"ID":"944334f2-f969-4e09-9bc4-36d669923e2a","Type":"ContainerDied","Data":"20315eee46db88f4e3bf025c5065a0be3b1dca77335bea6a509c8c425259a73d"}
Sep 04 22:05:39.585238 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.585206    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"]
Sep 04 22:05:39.588590 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.588567    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.590943 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.590912    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a5e1c2-serving-cert\""
Sep 04 22:05:39.603586 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.603561    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"]
Sep 04 22:05:39.703927 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.703905    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:39.727200 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.727171    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.727403 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.727231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.727403 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.727296    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-metrics-certs\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.727403 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.727325    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d76cm\" (UniqueName: \"kubernetes.io/projected/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-kube-api-access-d76cm\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.744420 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.744394    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7827-140431937424304_944334f2-f969-4e09-9bc4-36d669923e2a/test-curl-7827-140431937424304/0.log"
Sep 04 22:05:39.772366 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.772311    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7827-140431937424304"]
Sep 04 22:05:39.781487 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.781463    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7827-140431937424304"]
Sep 04 22:05:39.828643 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.828614    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgvrn\" (UniqueName: \"kubernetes.io/projected/944334f2-f969-4e09-9bc4-36d669923e2a-kube-api-access-mgvrn\") pod \"944334f2-f969-4e09-9bc4-36d669923e2a\" (UID: \"944334f2-f969-4e09-9bc4-36d669923e2a\") "
Sep 04 22:05:39.828812 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.828765    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.828895 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.828821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.828895 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.828854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-metrics-certs\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.828895 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.828881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d76cm\" (UniqueName: \"kubernetes.io/projected/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-kube-api-access-d76cm\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.829737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.829631    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.831031 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.830999    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/944334f2-f969-4e09-9bc4-36d669923e2a-kube-api-access-mgvrn" (OuterVolumeSpecName: "kube-api-access-mgvrn") pod "944334f2-f969-4e09-9bc4-36d669923e2a" (UID: "944334f2-f969-4e09-9bc4-36d669923e2a"). InnerVolumeSpecName "kube-api-access-mgvrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:05:39.831429 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.831406    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-metrics-certs\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.831512 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.831448    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.842082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.842047    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d76cm\" (UniqueName: \"kubernetes.io/projected/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-kube-api-access-d76cm\") pod \"maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.901726 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.901686    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:39.929820 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:39.929785    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgvrn\" (UniqueName: \"kubernetes.io/projected/944334f2-f969-4e09-9bc4-36d669923e2a-kube-api-access-mgvrn\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:05:40.174991 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.174962    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"]
Sep 04 22:05:40.176923 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:40.176897    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd34f2c5_b740_4708_9d07_03d0b8e6b2c3.slice/crio-43b326de701694f00bc540e3a447b9789426284de200abed1097f62c486004f8 WatchSource:0}: Error finding container 43b326de701694f00bc540e3a447b9789426284de200abed1097f62c486004f8: Status 404 returned error can't find the container with id 43b326de701694f00bc540e3a447b9789426284de200abed1097f62c486004f8
Sep 04 22:05:40.238652 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.238616    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"]
Sep 04 22:05:40.239015 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.238999    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944334f2-f969-4e09-9bc4-36d669923e2a" containerName="test-curl-7827-140431937424304"
Sep 04 22:05:40.239015 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.239015    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="944334f2-f969-4e09-9bc4-36d669923e2a" containerName="test-curl-7827-140431937424304"
Sep 04 22:05:40.239158 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.239091    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="944334f2-f969-4e09-9bc4-36d669923e2a" containerName="test-curl-7827-140431937424304"
Sep 04 22:05:40.243460 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.243434    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.247201 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.247087    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a5e1c2\""
Sep 04 22:05:40.250134 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.250105    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"]
Sep 04 22:05:40.253730 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.253709    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"]
Sep 04 22:05:40.253834 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.253813    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.256430 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.256407    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a5e1c2-dockercfg-ltcs6\""
Sep 04 22:05:40.261569 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.261548    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"]
Sep 04 22:05:40.339467 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.339424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbcq5\" (UniqueName: \"kubernetes.io/projected/84eecd63-5b01-4554-bad9-cc9f409f4232-kube-api-access-lbcq5\") pod \"payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.339634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.339522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84eecd63-5b01-4554-bad9-cc9f409f4232-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.339634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.339597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfhtq\" (UniqueName: \"kubernetes.io/projected/3684bc49-1570-4032-9ff0-268bf457d691-kube-api-access-nfhtq\") pod \"payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.339634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.339627    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3684bc49-1570-4032-9ff0-268bf457d691-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.440496 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.440384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84eecd63-5b01-4554-bad9-cc9f409f4232-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.440673 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.440513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfhtq\" (UniqueName: \"kubernetes.io/projected/3684bc49-1570-4032-9ff0-268bf457d691-kube-api-access-nfhtq\") pod \"payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.440673 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.440547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3684bc49-1570-4032-9ff0-268bf457d691-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.440673 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.440615    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbcq5\" (UniqueName: \"kubernetes.io/projected/84eecd63-5b01-4554-bad9-cc9f409f4232-kube-api-access-lbcq5\") pod \"payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.441205 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.441180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84eecd63-5b01-4554-bad9-cc9f409f4232-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.441332 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.441312    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3684bc49-1570-4032-9ff0-268bf457d691-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.449183 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.449163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfhtq\" (UniqueName: \"kubernetes.io/projected/3684bc49-1570-4032-9ff0-268bf457d691-kube-api-access-nfhtq\") pod \"payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.449602 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.449581    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbcq5\" (UniqueName: \"kubernetes.io/projected/84eecd63-5b01-4554-bad9-cc9f409f4232-kube-api-access-lbcq5\") pod \"payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.557126 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.557091    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:40.568225 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.567719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:40.573852 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.573823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8" event={"ID":"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3","Type":"ContainerStarted","Data":"43b326de701694f00bc540e3a447b9789426284de200abed1097f62c486004f8"}
Sep 04 22:05:40.578361 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.575409    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a562e1b54691d0bf85d730e480d21778160ed9e9f63fb96c05e3b622c868a1d"
Sep 04 22:05:40.578361 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.575496    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:40.591119 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:05:40.591092    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7827-140431937424304\" is forbidden: User \"system:node:ip-10-0-133-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-119.ec2.internal' and this object" podUID="944334f2-f969-4e09-9bc4-36d669923e2a" pod="opendatahub/test-curl-7827-140431937424304"
Sep 04 22:05:40.847007 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.846979    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"]
Sep 04 22:05:40.849043 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:40.849013    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84eecd63_5b01_4554_bad9_cc9f409f4232.slice/crio-08a0aeb1cf895742482cd2eab7e2282c035a7f75d41584f40c39d5bca6a427bb WatchSource:0}: Error finding container 08a0aeb1cf895742482cd2eab7e2282c035a7f75d41584f40c39d5bca6a427bb: Status 404 returned error can't find the container with id 08a0aeb1cf895742482cd2eab7e2282c035a7f75d41584f40c39d5bca6a427bb
Sep 04 22:05:40.864739 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:40.864704    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"]
Sep 04 22:05:40.869557 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:40.869528    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3684bc49_1570_4032_9ff0_268bf457d691.slice/crio-ce479739821ad6a0c5c04c4c6afdec88ed2e7a157778112f6653f442b13654de WatchSource:0}: Error finding container ce479739821ad6a0c5c04c4c6afdec88ed2e7a157778112f6653f442b13654de: Status 404 returned error can't find the container with id ce479739821ad6a0c5c04c4c6afdec88ed2e7a157778112f6653f442b13654de
Sep 04 22:05:41.391946 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.391913    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="944334f2-f969-4e09-9bc4-36d669923e2a" path="/var/lib/kubelet/pods/944334f2-f969-4e09-9bc4-36d669923e2a/volumes"
Sep 04 22:05:41.584138 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.584104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2" event={"ID":"3684bc49-1570-4032-9ff0-268bf457d691","Type":"ContainerStarted","Data":"6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5"}
Sep 04 22:05:41.584333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.584172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2" event={"ID":"3684bc49-1570-4032-9ff0-268bf457d691","Type":"ContainerStarted","Data":"ce479739821ad6a0c5c04c4c6afdec88ed2e7a157778112f6653f442b13654de"}
Sep 04 22:05:41.584333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.584253    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:41.585699 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.585676    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8" event={"ID":"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3","Type":"ContainerStarted","Data":"a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a"}
Sep 04 22:05:41.585818 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.585736    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:41.587202 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.587173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p" event={"ID":"84eecd63-5b01-4554-bad9-cc9f409f4232","Type":"ContainerStarted","Data":"1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434"}
Sep 04 22:05:41.587318 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.587214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p" event={"ID":"84eecd63-5b01-4554-bad9-cc9f409f4232","Type":"ContainerStarted","Data":"08a0aeb1cf895742482cd2eab7e2282c035a7f75d41584f40c39d5bca6a427bb"}
Sep 04 22:05:41.587318 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.587251    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:41.603179 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.603137    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2" podStartSLOduration=1.603124416 podStartE2EDuration="1.603124416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:05:41.601559866 +0000 UTC m=+716.685944725" watchObservedRunningTime="2026-09-04 22:05:41.603124416 +0000 UTC m=+716.687509257"
Sep 04 22:05:41.620831 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.620794    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8" podStartSLOduration=2.084945493 podStartE2EDuration="2.62078109s" totalImagesPullingTime="535.835597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:05:41.619183185 +0000 UTC m=+716.703568028" watchObservedRunningTime="2026-09-04 22:05:41.62078109 +0000 UTC m=+716.705165933"
Sep 04 22:05:41.636411 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:41.636361    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p" podStartSLOduration=1.636327247 podStartE2EDuration="1.636327247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:05:41.636026125 +0000 UTC m=+716.720410970" watchObservedRunningTime="2026-09-04 22:05:41.636327247 +0000 UTC m=+716.720712091"
Sep 04 22:05:45.033738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.033682    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"]
Sep 04 22:05:45.039159 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.039127    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.042016 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.041989    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f81019dc-serving-cert\""
Sep 04 22:05:45.051737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.051711    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"]
Sep 04 22:05:45.190597 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.190551    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-metrics-certs\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.190763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.190685    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfrq2\" (UniqueName: \"kubernetes.io/projected/6d2e44b6-54ff-4792-bac6-e40266a75478-kube-api-access-cfrq2\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.190763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.190759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.190860 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.190785    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2e44b6-54ff-4792-bac6-e40266a75478-openshift-service-ca\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.291781 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.291691    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.291781 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.291747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2e44b6-54ff-4792-bac6-e40266a75478-openshift-service-ca\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.292007 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.291792    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-metrics-certs\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.292007 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.291849    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfrq2\" (UniqueName: \"kubernetes.io/projected/6d2e44b6-54ff-4792-bac6-e40266a75478-kube-api-access-cfrq2\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.292505 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.292475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2e44b6-54ff-4792-bac6-e40266a75478-openshift-service-ca\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.294208 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.294188    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f81019dc-serving-cert\""
Sep 04 22:05:45.295205 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.295180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-metrics-certs\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.300526 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.300487    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfrq2\" (UniqueName: \"kubernetes.io/projected/6d2e44b6-54ff-4792-bac6-e40266a75478-kube-api-access-cfrq2\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.301925 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:05:45.301905    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-serving-cert: secret "maas-api-e2e-ait-f81019dc-serving-cert" not found
Sep 04 22:05:45.302019 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:05:45.301984    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls podName:6d2e44b6-54ff-4792-bac6-e40266a75478 nodeName:}" failed. No retries permitted until 2026-09-04 22:05:45.801961515 +0000 UTC m=+720.886346340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls") pod "maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" (UID: "6d2e44b6-54ff-4792-bac6-e40266a75478") : secret "maas-api-e2e-ait-f81019dc-serving-cert" not found
Sep 04 22:05:45.896521 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.896488    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.899070 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.899042    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls\") pod \"maas-api-e2e-ait-f81019dc-6894b85d76-srvcj\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:45.957131 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:45.957085    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:46.233442 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:46.233414    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"]
Sep 04 22:05:46.235606 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:05:46.235578    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2e44b6_54ff_4792_bac6_e40266a75478.slice/crio-5757ab77fe2297cb02da244c1e25216a555de7ad6588d280fe4d12735fe0777a WatchSource:0}: Error finding container 5757ab77fe2297cb02da244c1e25216a555de7ad6588d280fe4d12735fe0777a: Status 404 returned error can't find the container with id 5757ab77fe2297cb02da244c1e25216a555de7ad6588d280fe4d12735fe0777a
Sep 04 22:05:46.613688 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:46.613656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" event={"ID":"6d2e44b6-54ff-4792-bac6-e40266a75478","Type":"ContainerStarted","Data":"5757ab77fe2297cb02da244c1e25216a555de7ad6588d280fe4d12735fe0777a"}
Sep 04 22:05:47.597499 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:47.597467    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:05:47.620963 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:47.620927    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" event={"ID":"6d2e44b6-54ff-4792-bac6-e40266a75478","Type":"ContainerStarted","Data":"eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3"}
Sep 04 22:05:47.621117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:47.620995    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:05:47.638811 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:47.638739    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" podStartSLOduration=2.249173194 podStartE2EDuration="2.63871888s" totalImagesPullingTime="389.545686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:05:47.636574541 +0000 UTC m=+722.720959386" watchObservedRunningTime="2026-09-04 22:05:47.63871888 +0000 UTC m=+722.723103725"
Sep 04 22:05:52.594737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:52.594708    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:05:52.595233 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:52.594788    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:05:53.631950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:05:53.631922    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:06:08.755713 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.755679    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"]
Sep 04 22:06:08.761305 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.761282    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.763584 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.763565    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d4ad2d-serving-cert\""
Sep 04 22:06:08.772380 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.772352    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"]
Sep 04 22:06:08.889508 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.889472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-maas-api-tls\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.889643 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.889534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-metrics-certs\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.889643 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.889574    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13669269-3537-4266-9c94-2dbf0c372e25-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.889643 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.889597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcgrl\" (UniqueName: \"kubernetes.io/projected/13669269-3537-4266-9c94-2dbf0c372e25-kube-api-access-gcgrl\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.990630 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.990597    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-metrics-certs\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.990791 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.990643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13669269-3537-4266-9c94-2dbf0c372e25-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.990791 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.990676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcgrl\" (UniqueName: \"kubernetes.io/projected/13669269-3537-4266-9c94-2dbf0c372e25-kube-api-access-gcgrl\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.990791 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.990713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-maas-api-tls\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.991513 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.991400    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13669269-3537-4266-9c94-2dbf0c372e25-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.993246 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.993216    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-metrics-certs\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.993246 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.993230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-maas-api-tls\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:08.998108 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:08.998084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcgrl\" (UniqueName: \"kubernetes.io/projected/13669269-3537-4266-9c94-2dbf0c372e25-kube-api-access-gcgrl\") pod \"maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:09.072514 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.072451    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:09.354207 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.354182    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"]
Sep 04 22:06:09.356874 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:06:09.356848    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13669269_3537_4266_9c94_2dbf0c372e25.slice/crio-0ff4bc0ac42bfa54ed49e3f6298a81493cc145439d80e7a1d7cc10fee95b6eb8 WatchSource:0}: Error finding container 0ff4bc0ac42bfa54ed49e3f6298a81493cc145439d80e7a1d7cc10fee95b6eb8: Status 404 returned error can't find the container with id 0ff4bc0ac42bfa54ed49e3f6298a81493cc145439d80e7a1d7cc10fee95b6eb8
Sep 04 22:06:09.424003 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.423976    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"]
Sep 04 22:06:09.427687 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.427664    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.430975 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.430955    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d4ad2d\""
Sep 04 22:06:09.444263 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.444242    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"]
Sep 04 22:06:09.495406 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.495375    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ftxs\" (UniqueName: \"kubernetes.io/projected/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-kube-api-access-8ftxs\") pod \"payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.495552 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.495512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.596308 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.596276    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.596494 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.596322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ftxs\" (UniqueName: \"kubernetes.io/projected/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-kube-api-access-8ftxs\") pod \"payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.596978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.596957    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.605485 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.605433    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ftxs\" (UniqueName: \"kubernetes.io/projected/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-kube-api-access-8ftxs\") pod \"payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:09.730982 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.730946    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" event={"ID":"13669269-3537-4266-9c94-2dbf0c372e25","Type":"ContainerStarted","Data":"0ff4bc0ac42bfa54ed49e3f6298a81493cc145439d80e7a1d7cc10fee95b6eb8"}
Sep 04 22:06:09.738146 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:09.738120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:10.017682 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.017660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"]
Sep 04 22:06:10.019824 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:06:10.019797    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61a2dbf_630c_4ee6_b9ab_cb472b68e16c.slice/crio-4288eb33b2d4cc979d0281ffce8013a135c8e5a1f27bc619c417ce00d19dea5a WatchSource:0}: Error finding container 4288eb33b2d4cc979d0281ffce8013a135c8e5a1f27bc619c417ce00d19dea5a: Status 404 returned error can't find the container with id 4288eb33b2d4cc979d0281ffce8013a135c8e5a1f27bc619c417ce00d19dea5a
Sep 04 22:06:10.737634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.737576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" event={"ID":"13669269-3537-4266-9c94-2dbf0c372e25","Type":"ContainerStarted","Data":"22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618"}
Sep 04 22:06:10.737806 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.737669    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:10.739225 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.739199    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr" event={"ID":"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c","Type":"ContainerStarted","Data":"711f41d635972090e4cee47a93ede589da39a251f23e24e0cb223e0755133a3a"}
Sep 04 22:06:10.739381 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.739245    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr" event={"ID":"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c","Type":"ContainerStarted","Data":"4288eb33b2d4cc979d0281ffce8013a135c8e5a1f27bc619c417ce00d19dea5a"}
Sep 04 22:06:10.739381 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.739316    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:10.755795 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.755752    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" podStartSLOduration=2.31794715 podStartE2EDuration="2.755736698s" totalImagesPullingTime="437.789548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:10.753927457 +0000 UTC m=+745.838312302" watchObservedRunningTime="2026-09-04 22:06:10.755736698 +0000 UTC m=+745.840121543"
Sep 04 22:06:10.770188 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:10.770143    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr" podStartSLOduration=1.770129504 podStartE2EDuration="1.770129504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:10.769482164 +0000 UTC m=+745.853867009" watchObservedRunningTime="2026-09-04 22:06:10.770129504 +0000 UTC m=+745.854514348"
Sep 04 22:06:16.749780 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:16.749745    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:06:21.747190 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:21.747155    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:06:25.918290 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:25.918253    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"]
Sep 04 22:06:25.918690 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:25.918508    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" podUID="6d2e44b6-54ff-4792-bac6-e40266a75478" containerName="maas-api" containerID="cri-o://eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3" gracePeriod=30
Sep 04 22:06:26.162813 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.162788    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:06:26.245462 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.245383    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfrq2\" (UniqueName: \"kubernetes.io/projected/6d2e44b6-54ff-4792-bac6-e40266a75478-kube-api-access-cfrq2\") pod \"6d2e44b6-54ff-4792-bac6-e40266a75478\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") "
Sep 04 22:06:26.245628 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.245481    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2e44b6-54ff-4792-bac6-e40266a75478-openshift-service-ca\") pod \"6d2e44b6-54ff-4792-bac6-e40266a75478\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") "
Sep 04 22:06:26.245628 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.245560    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-metrics-certs\") pod \"6d2e44b6-54ff-4792-bac6-e40266a75478\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") "
Sep 04 22:06:26.245628 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.245579    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls\") pod \"6d2e44b6-54ff-4792-bac6-e40266a75478\" (UID: \"6d2e44b6-54ff-4792-bac6-e40266a75478\") "
Sep 04 22:06:26.246138 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.246112    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2e44b6-54ff-4792-bac6-e40266a75478-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d2e44b6-54ff-4792-bac6-e40266a75478" (UID: "6d2e44b6-54ff-4792-bac6-e40266a75478"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:06:26.248824 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.248785    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6d2e44b6-54ff-4792-bac6-e40266a75478" (UID: "6d2e44b6-54ff-4792-bac6-e40266a75478"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:06:26.250492 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.250446    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2e44b6-54ff-4792-bac6-e40266a75478-kube-api-access-cfrq2" (OuterVolumeSpecName: "kube-api-access-cfrq2") pod "6d2e44b6-54ff-4792-bac6-e40266a75478" (UID: "6d2e44b6-54ff-4792-bac6-e40266a75478"). InnerVolumeSpecName "kube-api-access-cfrq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:06:26.250597 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.250557    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d2e44b6-54ff-4792-bac6-e40266a75478" (UID: "6d2e44b6-54ff-4792-bac6-e40266a75478"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:06:26.346961 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.346924    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2e44b6-54ff-4792-bac6-e40266a75478-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:06:26.346961 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.346955    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:06:26.346961 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.346966    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2e44b6-54ff-4792-bac6-e40266a75478-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:06:26.347182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.346973    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfrq2\" (UniqueName: \"kubernetes.io/projected/6d2e44b6-54ff-4792-bac6-e40266a75478-kube-api-access-cfrq2\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:06:26.824751 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.824716    2565 generic.go:356] "Generic (PLEG): container finished" podID="6d2e44b6-54ff-4792-bac6-e40266a75478" containerID="eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3" exitCode=0
Sep 04 22:06:26.824751 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.824753    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" event={"ID":"6d2e44b6-54ff-4792-bac6-e40266a75478","Type":"ContainerDied","Data":"eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3"}
Sep 04 22:06:26.824950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.824762    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"
Sep 04 22:06:26.824950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.824792    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj" event={"ID":"6d2e44b6-54ff-4792-bac6-e40266a75478","Type":"ContainerDied","Data":"5757ab77fe2297cb02da244c1e25216a555de7ad6588d280fe4d12735fe0777a"}
Sep 04 22:06:26.824950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.824812    2565 scope.go:117] "RemoveContainer" containerID="eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3"
Sep 04 22:06:26.833612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.833595    2565 scope.go:117] "RemoveContainer" containerID="eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3"
Sep 04 22:06:26.833873 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:06:26.833855    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3\": container with ID starting with eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3 not found: ID does not exist" containerID="eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3"
Sep 04 22:06:26.833928 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.833883    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3"} err="failed to get container status \"eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3\": rpc error: code = NotFound desc = could not find container \"eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3\": container with ID starting with eb77720269b7d723469d918c6fb6f19fc25b4e17e0f3ae3e8fedeef6f52f2cf3 not found: ID does not exist"
Sep 04 22:06:26.850734 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.850711    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"]
Sep 04 22:06:26.857595 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:26.857575    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f81019dc-6894b85d76-srvcj"]
Sep 04 22:06:27.390309 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:27.390275    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d2e44b6-54ff-4792-bac6-e40266a75478" path="/var/lib/kubelet/pods/6d2e44b6-54ff-4792-bac6-e40266a75478/volumes"
Sep 04 22:06:30.738209 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.738177    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"]
Sep 04 22:06:30.738697 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.738678    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d2e44b6-54ff-4792-bac6-e40266a75478" containerName="maas-api"
Sep 04 22:06:30.738697 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.738692    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d2e44b6-54ff-4792-bac6-e40266a75478" containerName="maas-api"
Sep 04 22:06:30.738804 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.738754    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d2e44b6-54ff-4792-bac6-e40266a75478" containerName="maas-api"
Sep 04 22:06:30.743183 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.743157    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.745405 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.745384    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6fda89ac-serving-cert\""
Sep 04 22:06:30.750231 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.750203    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"]
Sep 04 22:06:30.885923 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.885886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-metrics-certs\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.885923 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.885922    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2145682d-bf40-457a-a7dd-1a0dea07f16e-openshift-service-ca\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.886145 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.885991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-maas-api-tls\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.886145 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.886036    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74llg\" (UniqueName: \"kubernetes.io/projected/2145682d-bf40-457a-a7dd-1a0dea07f16e-kube-api-access-74llg\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.987153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.987116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-maas-api-tls\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.987333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.987175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74llg\" (UniqueName: \"kubernetes.io/projected/2145682d-bf40-457a-a7dd-1a0dea07f16e-kube-api-access-74llg\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.987333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.987216    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-metrics-certs\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.987333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.987234    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2145682d-bf40-457a-a7dd-1a0dea07f16e-openshift-service-ca\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.988240 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.988183    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2145682d-bf40-457a-a7dd-1a0dea07f16e-openshift-service-ca\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.989689 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.989661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-metrics-certs\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:30.989799 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:30.989776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-maas-api-tls\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:31.001655 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.001631    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74llg\" (UniqueName: \"kubernetes.io/projected/2145682d-bf40-457a-a7dd-1a0dea07f16e-kube-api-access-74llg\") pod \"maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:31.056693 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.056654    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:31.327101 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.327075    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"]
Sep 04 22:06:31.329896 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:06:31.329870    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2145682d_bf40_457a_a7dd_1a0dea07f16e.slice/crio-6f0093aaba7315658f4df89e2f9b3d5795802b69d6e2f7941128caa9308591ff WatchSource:0}: Error finding container 6f0093aaba7315658f4df89e2f9b3d5795802b69d6e2f7941128caa9308591ff: Status 404 returned error can't find the container with id 6f0093aaba7315658f4df89e2f9b3d5795802b69d6e2f7941128caa9308591ff
Sep 04 22:06:31.850754 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.850722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn" event={"ID":"2145682d-bf40-457a-a7dd-1a0dea07f16e","Type":"ContainerStarted","Data":"a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2"}
Sep 04 22:06:31.850754 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.850773    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn" event={"ID":"2145682d-bf40-457a-a7dd-1a0dea07f16e","Type":"ContainerStarted","Data":"6f0093aaba7315658f4df89e2f9b3d5795802b69d6e2f7941128caa9308591ff"}
Sep 04 22:06:31.851242 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.850800    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:06:31.877748 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:31.877689    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn" podStartSLOduration=1.548400971 podStartE2EDuration="1.877670877s" totalImagesPullingTime="329.269906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:31.875738522 +0000 UTC m=+766.960123367" watchObservedRunningTime="2026-09-04 22:06:31.877670877 +0000 UTC m=+766.962055724"
Sep 04 22:06:37.862888 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:06:37.862850    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:07:01.945407 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:01.945333    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"]
Sep 04 22:07:01.946014 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:01.945629    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn" podUID="2145682d-bf40-457a-a7dd-1a0dea07f16e" containerName="maas-api" containerID="cri-o://a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2" gracePeriod=30
Sep 04 22:07:02.203960 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.203899    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:07:02.265062 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.265024    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74llg\" (UniqueName: \"kubernetes.io/projected/2145682d-bf40-457a-a7dd-1a0dea07f16e-kube-api-access-74llg\") pod \"2145682d-bf40-457a-a7dd-1a0dea07f16e\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") "
Sep 04 22:07:02.265232 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.265080    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2145682d-bf40-457a-a7dd-1a0dea07f16e-openshift-service-ca\") pod \"2145682d-bf40-457a-a7dd-1a0dea07f16e\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") "
Sep 04 22:07:02.265232 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.265147    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-metrics-certs\") pod \"2145682d-bf40-457a-a7dd-1a0dea07f16e\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") "
Sep 04 22:07:02.265373 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.265264    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-maas-api-tls\") pod \"2145682d-bf40-457a-a7dd-1a0dea07f16e\" (UID: \"2145682d-bf40-457a-a7dd-1a0dea07f16e\") "
Sep 04 22:07:02.265522 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.265484    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2145682d-bf40-457a-a7dd-1a0dea07f16e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2145682d-bf40-457a-a7dd-1a0dea07f16e" (UID: "2145682d-bf40-457a-a7dd-1a0dea07f16e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:07:02.265646 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.265590    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2145682d-bf40-457a-a7dd-1a0dea07f16e-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.267517 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.267486    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2145682d-bf40-457a-a7dd-1a0dea07f16e-kube-api-access-74llg" (OuterVolumeSpecName: "kube-api-access-74llg") pod "2145682d-bf40-457a-a7dd-1a0dea07f16e" (UID: "2145682d-bf40-457a-a7dd-1a0dea07f16e"). InnerVolumeSpecName "kube-api-access-74llg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:07:02.267623 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.267534    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2145682d-bf40-457a-a7dd-1a0dea07f16e" (UID: "2145682d-bf40-457a-a7dd-1a0dea07f16e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:07:02.267772 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.267755    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2145682d-bf40-457a-a7dd-1a0dea07f16e" (UID: "2145682d-bf40-457a-a7dd-1a0dea07f16e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:07:02.366604 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.366573    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.366604 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.366604    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2145682d-bf40-457a-a7dd-1a0dea07f16e-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.366834 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:02.366616    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74llg\" (UniqueName: \"kubernetes.io/projected/2145682d-bf40-457a-a7dd-1a0dea07f16e-kube-api-access-74llg\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:03.007282 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.007248    2565 generic.go:356] "Generic (PLEG): container finished" podID="2145682d-bf40-457a-a7dd-1a0dea07f16e" containerID="a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2" exitCode=0
Sep 04 22:07:03.007738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.007295    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"
Sep 04 22:07:03.007738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.007329    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn" event={"ID":"2145682d-bf40-457a-a7dd-1a0dea07f16e","Type":"ContainerDied","Data":"a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2"}
Sep 04 22:07:03.007738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.007410    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn" event={"ID":"2145682d-bf40-457a-a7dd-1a0dea07f16e","Type":"ContainerDied","Data":"6f0093aaba7315658f4df89e2f9b3d5795802b69d6e2f7941128caa9308591ff"}
Sep 04 22:07:03.007738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.007442    2565 scope.go:117] "RemoveContainer" containerID="a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2"
Sep 04 22:07:03.016925 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.016908    2565 scope.go:117] "RemoveContainer" containerID="a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2"
Sep 04 22:07:03.017193 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:07:03.017177    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2\": container with ID starting with a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2 not found: ID does not exist" containerID="a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2"
Sep 04 22:07:03.017258 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.017199    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2"} err="failed to get container status \"a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2\": rpc error: code = NotFound desc = could not find container \"a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2\": container with ID starting with a27111715b65bd03f63b0a99ce2f1406b59fca5e4a2d2065589125747e0c01d2 not found: ID does not exist"
Sep 04 22:07:03.028958 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.028938    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"]
Sep 04 22:07:03.043115 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.043092    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6fda89ac-7794f4b59c-j57cn"]
Sep 04 22:07:03.389903 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:03.389869    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2145682d-bf40-457a-a7dd-1a0dea07f16e" path="/var/lib/kubelet/pods/2145682d-bf40-457a-a7dd-1a0dea07f16e/volumes"
Sep 04 22:07:06.746375 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.746278    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"]
Sep 04 22:07:06.746966 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.746947    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2145682d-bf40-457a-a7dd-1a0dea07f16e" containerName="maas-api"
Sep 04 22:07:06.747065 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.746968    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2145682d-bf40-457a-a7dd-1a0dea07f16e" containerName="maas-api"
Sep 04 22:07:06.747065 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.747053    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2145682d-bf40-457a-a7dd-1a0dea07f16e" containerName="maas-api"
Sep 04 22:07:06.752543 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.752520    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.754749 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.754731    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a43b7e35-serving-cert\""
Sep 04 22:07:06.761399 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.761379    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"]
Sep 04 22:07:06.808207 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.808177    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-metrics-certs\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.808396 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.808216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db2fb\" (UniqueName: \"kubernetes.io/projected/82acc3e4-b1da-4fb9-8d48-5d7047381623-kube-api-access-db2fb\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.808396 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.808241    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.808396 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.808275    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acc3e4-b1da-4fb9-8d48-5d7047381623-openshift-service-ca\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.909655 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.909619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-metrics-certs\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.909655 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.909660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db2fb\" (UniqueName: \"kubernetes.io/projected/82acc3e4-b1da-4fb9-8d48-5d7047381623-kube-api-access-db2fb\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.909894 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.909682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.909894 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.909719    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acc3e4-b1da-4fb9-8d48-5d7047381623-openshift-service-ca\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.909894 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:07:06.909791    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-serving-cert: secret "maas-api-e2e-ait-a43b7e35-serving-cert" not found
Sep 04 22:07:06.909894 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:07:06.909856    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls podName:82acc3e4-b1da-4fb9-8d48-5d7047381623 nodeName:}" failed. No retries permitted until 2026-09-04 22:07:07.409840026 +0000 UTC m=+802.494224849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls") pod "maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" (UID: "82acc3e4-b1da-4fb9-8d48-5d7047381623") : secret "maas-api-e2e-ait-a43b7e35-serving-cert" not found
Sep 04 22:07:06.910475 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.910456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acc3e4-b1da-4fb9-8d48-5d7047381623-openshift-service-ca\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.912368 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.912324    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-metrics-certs\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:06.918386 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:06.918331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db2fb\" (UniqueName: \"kubernetes.io/projected/82acc3e4-b1da-4fb9-8d48-5d7047381623-kube-api-access-db2fb\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:07.413753 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:07.413713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:07.416314 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:07.416281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls\") pod \"maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:07.664618 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:07.664484    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:07.941571 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:07.941540    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"]
Sep 04 22:07:07.943141 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:07:07.943116    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82acc3e4_b1da_4fb9_8d48_5d7047381623.slice/crio-5527502eeaa8efea98939c9a16b0083db0e45044cab5a117a0908fe3d3fb0b3f WatchSource:0}: Error finding container 5527502eeaa8efea98939c9a16b0083db0e45044cab5a117a0908fe3d3fb0b3f: Status 404 returned error can't find the container with id 5527502eeaa8efea98939c9a16b0083db0e45044cab5a117a0908fe3d3fb0b3f
Sep 04 22:07:08.033646 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:08.033616    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" event={"ID":"82acc3e4-b1da-4fb9-8d48-5d7047381623","Type":"ContainerStarted","Data":"5527502eeaa8efea98939c9a16b0083db0e45044cab5a117a0908fe3d3fb0b3f"}
Sep 04 22:07:09.039862 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:09.039818    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" event={"ID":"82acc3e4-b1da-4fb9-8d48-5d7047381623","Type":"ContainerStarted","Data":"3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f"}
Sep 04 22:07:09.040364 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:09.039912    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:09.058329 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:09.058283    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" podStartSLOduration=2.700037735 podStartE2EDuration="3.05827022s" totalImagesPullingTime="358.232485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:09.056520857 +0000 UTC m=+804.140905700" watchObservedRunningTime="2026-09-04 22:07:09.05827022 +0000 UTC m=+804.142655080"
Sep 04 22:07:15.050805 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:15.050777    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:24.033147 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.033107    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"]
Sep 04 22:07:24.037301 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.037282    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.040464 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.040443    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d4ad2d\"/\"kube-root-ca.crt\""
Sep 04 22:07:24.040612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.040453    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d4ad2d\"/\"test-model-d4ad2d-kserve-self-signed-certs\""
Sep 04 22:07:24.040807 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.040556    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d4ad2d\"/\"openshift-service-ca.crt\""
Sep 04 22:07:24.041117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.040594    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d4ad2d\"/\"default-dockercfg-srfkk\""
Sep 04 22:07:24.045752 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.045727    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"]
Sep 04 22:07:24.167204 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.167167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-dshm\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.167405 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.167225    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-home\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.167405 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.167290    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tmp-dir\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.167405 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.167331    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-model-cache\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.167405 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.167377    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9gsz\" (UniqueName: \"kubernetes.io/projected/b1a74a38-7500-4c68-805d-5eebe2aa9f92-kube-api-access-n9gsz\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.167560 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.167411    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tls-certs\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268130 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-dshm\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268130 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268096    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-home\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268130 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268125    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tmp-dir\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268423 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-model-cache\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268423 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9gsz\" (UniqueName: \"kubernetes.io/projected/b1a74a38-7500-4c68-805d-5eebe2aa9f92-kube-api-access-n9gsz\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268423 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268223    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tls-certs\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268606 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268581    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-home\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268668 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268635    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tmp-dir\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.268668 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.268655    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-model-cache\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.270587 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.270571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-dshm\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.270747 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.270732    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tls-certs\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.276447 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.276420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9gsz\" (UniqueName: \"kubernetes.io/projected/b1a74a38-7500-4c68-805d-5eebe2aa9f92-kube-api-access-n9gsz\") pod \"test-model-d4ad2d-kserve-764b5dbdf9-4cb29\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") " pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.349482 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.349453    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:24.621311 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:24.621287    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"]
Sep 04 22:07:24.623411 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:07:24.623387    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1a74a38_7500_4c68_805d_5eebe2aa9f92.slice/crio-08c2a98a0b0148692a85a1ad75592b4f28f17317c494b34b1c493af36b0ec0c5 WatchSource:0}: Error finding container 08c2a98a0b0148692a85a1ad75592b4f28f17317c494b34b1c493af36b0ec0c5: Status 404 returned error can't find the container with id 08c2a98a0b0148692a85a1ad75592b4f28f17317c494b34b1c493af36b0ec0c5
Sep 04 22:07:25.120704 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:25.120664    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29" event={"ID":"b1a74a38-7500-4c68-805d-5eebe2aa9f92","Type":"ContainerStarted","Data":"907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229"}
Sep 04 22:07:25.120704 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:25.120707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29" event={"ID":"b1a74a38-7500-4c68-805d-5eebe2aa9f92","Type":"ContainerStarted","Data":"08c2a98a0b0148692a85a1ad75592b4f28f17317c494b34b1c493af36b0ec0c5"}
Sep 04 22:07:25.143080 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:25.143035    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29" podStartSLOduration=1.143020817 podStartE2EDuration="1.143020817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:25.141628283 +0000 UTC m=+820.226013126" watchObservedRunningTime="2026-09-04 22:07:25.143020817 +0000 UTC m=+820.227405662"
Sep 04 22:07:34.349574 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:34.349541    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:34.350155 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:34.349593    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:34.362352 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:34.362311    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:35.181192 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:35.181162    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:07:37.169984 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.169948    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"]
Sep 04 22:07:37.170482 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.170240    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" podUID="82acc3e4-b1da-4fb9-8d48-5d7047381623" containerName="maas-api" containerID="cri-o://3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f" gracePeriod=30
Sep 04 22:07:37.420217 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.420163    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:37.488870 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.488835    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db2fb\" (UniqueName: \"kubernetes.io/projected/82acc3e4-b1da-4fb9-8d48-5d7047381623-kube-api-access-db2fb\") pod \"82acc3e4-b1da-4fb9-8d48-5d7047381623\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") "
Sep 04 22:07:37.489042 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.488925    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-metrics-certs\") pod \"82acc3e4-b1da-4fb9-8d48-5d7047381623\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") "
Sep 04 22:07:37.489042 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.488958    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls\") pod \"82acc3e4-b1da-4fb9-8d48-5d7047381623\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") "
Sep 04 22:07:37.489042 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.489004    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acc3e4-b1da-4fb9-8d48-5d7047381623-openshift-service-ca\") pod \"82acc3e4-b1da-4fb9-8d48-5d7047381623\" (UID: \"82acc3e4-b1da-4fb9-8d48-5d7047381623\") "
Sep 04 22:07:37.489388 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.489363    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82acc3e4-b1da-4fb9-8d48-5d7047381623-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82acc3e4-b1da-4fb9-8d48-5d7047381623" (UID: "82acc3e4-b1da-4fb9-8d48-5d7047381623"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:07:37.491036 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.490999    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82acc3e4-b1da-4fb9-8d48-5d7047381623" (UID: "82acc3e4-b1da-4fb9-8d48-5d7047381623"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:07:37.491133 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.491054    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "82acc3e4-b1da-4fb9-8d48-5d7047381623" (UID: "82acc3e4-b1da-4fb9-8d48-5d7047381623"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:07:37.491133 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.491068    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82acc3e4-b1da-4fb9-8d48-5d7047381623-kube-api-access-db2fb" (OuterVolumeSpecName: "kube-api-access-db2fb") pod "82acc3e4-b1da-4fb9-8d48-5d7047381623" (UID: "82acc3e4-b1da-4fb9-8d48-5d7047381623"). InnerVolumeSpecName "kube-api-access-db2fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:07:37.589904 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.589874    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db2fb\" (UniqueName: \"kubernetes.io/projected/82acc3e4-b1da-4fb9-8d48-5d7047381623-kube-api-access-db2fb\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:37.589904 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.589900    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:37.589904 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.589909    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82acc3e4-b1da-4fb9-8d48-5d7047381623-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:37.590103 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:37.589917    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82acc3e4-b1da-4fb9-8d48-5d7047381623-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:07:38.188131 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.188094    2565 generic.go:356] "Generic (PLEG): container finished" podID="82acc3e4-b1da-4fb9-8d48-5d7047381623" containerID="3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f" exitCode=0
Sep 04 22:07:38.188536 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.188140    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"
Sep 04 22:07:38.188536 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.188177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" event={"ID":"82acc3e4-b1da-4fb9-8d48-5d7047381623","Type":"ContainerDied","Data":"3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f"}
Sep 04 22:07:38.188536 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.188234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc" event={"ID":"82acc3e4-b1da-4fb9-8d48-5d7047381623","Type":"ContainerDied","Data":"5527502eeaa8efea98939c9a16b0083db0e45044cab5a117a0908fe3d3fb0b3f"}
Sep 04 22:07:38.188536 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.188260    2565 scope.go:117] "RemoveContainer" containerID="3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f"
Sep 04 22:07:38.198099 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.198082    2565 scope.go:117] "RemoveContainer" containerID="3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f"
Sep 04 22:07:38.198379 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:07:38.198360    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f\": container with ID starting with 3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f not found: ID does not exist" containerID="3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f"
Sep 04 22:07:38.198425 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.198388    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f"} err="failed to get container status \"3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f\": rpc error: code = NotFound desc = could not find container \"3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f\": container with ID starting with 3150c35b5943656843f619613b7d58ff73adb32897c47d83b5623e541bcd490f not found: ID does not exist"
Sep 04 22:07:38.213478 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.213451    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"]
Sep 04 22:07:38.216707 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:38.216686    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a43b7e35-fbb8db48d-49rhc"]
Sep 04 22:07:39.390221 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:39.390179    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82acc3e4-b1da-4fb9-8d48-5d7047381623" path="/var/lib/kubelet/pods/82acc3e4-b1da-4fb9-8d48-5d7047381623/volumes"
Sep 04 22:07:43.048422 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.048376    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"]
Sep 04 22:07:43.049064 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.049044    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82acc3e4-b1da-4fb9-8d48-5d7047381623" containerName="maas-api"
Sep 04 22:07:43.049131 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.049067    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="82acc3e4-b1da-4fb9-8d48-5d7047381623" containerName="maas-api"
Sep 04 22:07:43.049192 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.049182    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="82acc3e4-b1da-4fb9-8d48-5d7047381623" containerName="maas-api"
Sep 04 22:07:43.052308 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.052285    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.055085 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.055061    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-5d398f01-serving-cert\""
Sep 04 22:07:43.065492 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.065470    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"]
Sep 04 22:07:43.140653 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.140617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a745779-df57-4a82-9b62-54feaaf3f2a4-openshift-service-ca\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.140827 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.140671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-maas-api-tls\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.140827 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.140737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-metrics-certs\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.140827 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.140758    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b8k9\" (UniqueName: \"kubernetes.io/projected/9a745779-df57-4a82-9b62-54feaaf3f2a4-kube-api-access-4b8k9\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.241701 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.241665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-metrics-certs\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.241701 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.241712    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b8k9\" (UniqueName: \"kubernetes.io/projected/9a745779-df57-4a82-9b62-54feaaf3f2a4-kube-api-access-4b8k9\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.242012 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.241761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a745779-df57-4a82-9b62-54feaaf3f2a4-openshift-service-ca\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.242012 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.241815    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-maas-api-tls\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.242456 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.242434    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a745779-df57-4a82-9b62-54feaaf3f2a4-openshift-service-ca\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.244232 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.244210    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-metrics-certs\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.244324 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.244259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-maas-api-tls\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.254792 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.254764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b8k9\" (UniqueName: \"kubernetes.io/projected/9a745779-df57-4a82-9b62-54feaaf3f2a4-kube-api-access-4b8k9\") pod \"maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.363131 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.363103    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:43.641088 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:43.641063    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"]
Sep 04 22:07:43.643780 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:07:43.643743    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a745779_df57_4a82_9b62_54feaaf3f2a4.slice/crio-a96fd37e0496baac6dbb6c18f74f6300d49cb237b19e6c2cbf936e4bec8174f1 WatchSource:0}: Error finding container a96fd37e0496baac6dbb6c18f74f6300d49cb237b19e6c2cbf936e4bec8174f1: Status 404 returned error can't find the container with id a96fd37e0496baac6dbb6c18f74f6300d49cb237b19e6c2cbf936e4bec8174f1
Sep 04 22:07:44.220591 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:44.220565    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd" event={"ID":"9a745779-df57-4a82-9b62-54feaaf3f2a4","Type":"ContainerStarted","Data":"a96fd37e0496baac6dbb6c18f74f6300d49cb237b19e6c2cbf936e4bec8174f1"}
Sep 04 22:07:45.227159 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:45.227109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd" event={"ID":"9a745779-df57-4a82-9b62-54feaaf3f2a4","Type":"ContainerStarted","Data":"346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040"}
Sep 04 22:07:45.227571 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:45.227240    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:07:45.245522 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:45.245476    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd" podStartSLOduration=1.726195171 podStartE2EDuration="2.245461735s" totalImagesPullingTime="519.266564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:45.24438117 +0000 UTC m=+840.328766010" watchObservedRunningTime="2026-09-04 22:07:45.245461735 +0000 UTC m=+840.329846579"
Sep 04 22:07:51.238364 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:07:51.238309    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:08:13.615827 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:13.615787    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"]
Sep 04 22:08:13.616773 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:13.616714    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd" podUID="9a745779-df57-4a82-9b62-54feaaf3f2a4" containerName="maas-api" containerID="cri-o://346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040" gracePeriod=30
Sep 04 22:08:13.870860 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:13.870806    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:08:14.024495 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.024460    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a745779-df57-4a82-9b62-54feaaf3f2a4-openshift-service-ca\") pod \"9a745779-df57-4a82-9b62-54feaaf3f2a4\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") "
Sep 04 22:08:14.024676 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.024511    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-metrics-certs\") pod \"9a745779-df57-4a82-9b62-54feaaf3f2a4\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") "
Sep 04 22:08:14.024676 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.024592    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b8k9\" (UniqueName: \"kubernetes.io/projected/9a745779-df57-4a82-9b62-54feaaf3f2a4-kube-api-access-4b8k9\") pod \"9a745779-df57-4a82-9b62-54feaaf3f2a4\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") "
Sep 04 22:08:14.024676 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.024643    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-maas-api-tls\") pod \"9a745779-df57-4a82-9b62-54feaaf3f2a4\" (UID: \"9a745779-df57-4a82-9b62-54feaaf3f2a4\") "
Sep 04 22:08:14.024944 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.024912    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a745779-df57-4a82-9b62-54feaaf3f2a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a745779-df57-4a82-9b62-54feaaf3f2a4" (UID: "9a745779-df57-4a82-9b62-54feaaf3f2a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:08:14.026983 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.026953    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9a745779-df57-4a82-9b62-54feaaf3f2a4" (UID: "9a745779-df57-4a82-9b62-54feaaf3f2a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:08:14.027445 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.027418    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a745779-df57-4a82-9b62-54feaaf3f2a4-kube-api-access-4b8k9" (OuterVolumeSpecName: "kube-api-access-4b8k9") pod "9a745779-df57-4a82-9b62-54feaaf3f2a4" (UID: "9a745779-df57-4a82-9b62-54feaaf3f2a4"). InnerVolumeSpecName "kube-api-access-4b8k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:08:14.027445 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.027422    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a745779-df57-4a82-9b62-54feaaf3f2a4" (UID: "9a745779-df57-4a82-9b62-54feaaf3f2a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:08:14.126124 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.126046    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.126124 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.126074    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b8k9\" (UniqueName: \"kubernetes.io/projected/9a745779-df57-4a82-9b62-54feaaf3f2a4-kube-api-access-4b8k9\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.126124 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.126084    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a745779-df57-4a82-9b62-54feaaf3f2a4-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.126124 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.126093    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a745779-df57-4a82-9b62-54feaaf3f2a4-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.381597 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.381512    2565 generic.go:356] "Generic (PLEG): container finished" podID="9a745779-df57-4a82-9b62-54feaaf3f2a4" containerID="346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040" exitCode=0
Sep 04 22:08:14.381597 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.381563    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"
Sep 04 22:08:14.381802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.381597    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd" event={"ID":"9a745779-df57-4a82-9b62-54feaaf3f2a4","Type":"ContainerDied","Data":"346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040"}
Sep 04 22:08:14.381802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.381651    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd" event={"ID":"9a745779-df57-4a82-9b62-54feaaf3f2a4","Type":"ContainerDied","Data":"a96fd37e0496baac6dbb6c18f74f6300d49cb237b19e6c2cbf936e4bec8174f1"}
Sep 04 22:08:14.381802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.381672    2565 scope.go:117] "RemoveContainer" containerID="346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040"
Sep 04 22:08:14.391140 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.391124    2565 scope.go:117] "RemoveContainer" containerID="346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040"
Sep 04 22:08:14.391408 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:08:14.391389    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040\": container with ID starting with 346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040 not found: ID does not exist" containerID="346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040"
Sep 04 22:08:14.391460 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.391417    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040"} err="failed to get container status \"346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040\": rpc error: code = NotFound desc = could not find container \"346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040\": container with ID starting with 346f908c43771718c264e89bbc30908c22971eb674256bd282864f521960b040 not found: ID does not exist"
Sep 04 22:08:14.402829 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.402805    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"]
Sep 04 22:08:14.406802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:14.406783    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5d398f01-554d8c65b7-rqcwd"]
Sep 04 22:08:15.390359 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:15.390313    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a745779-df57-4a82-9b62-54feaaf3f2a4" path="/var/lib/kubelet/pods/9a745779-df57-4a82-9b62-54feaaf3f2a4/volumes"
Sep 04 22:08:53.029130 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:08:53.029096    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"]
Sep 04 22:09:03.579149 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.579111    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"]
Sep 04 22:09:03.579737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.579380    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8" podUID="bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" containerName="maas-api" containerID="cri-o://a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a" gracePeriod=30
Sep 04 22:09:03.645122 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.645082    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"]
Sep 04 22:09:03.645382 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.645322    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2" podUID="3684bc49-1570-4032-9ff0-268bf457d691" containerName="payload-processing" containerID="cri-o://6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5" gracePeriod=30
Sep 04 22:09:03.651369 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.651327    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"]
Sep 04 22:09:03.651722 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.651694    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p" podUID="84eecd63-5b01-4554-bad9-cc9f409f4232" containerName="payload-pre-processing" containerID="cri-o://1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434" gracePeriod=30
Sep 04 22:09:03.938085 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.938061    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:09:03.940938 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.940919    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:09:03.944249 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:03.944231    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:09:04.087018 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.086980    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfhtq\" (UniqueName: \"kubernetes.io/projected/3684bc49-1570-4032-9ff0-268bf457d691-kube-api-access-nfhtq\") pod \"3684bc49-1570-4032-9ff0-268bf457d691\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") "
Sep 04 22:09:04.087208 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087037    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d76cm\" (UniqueName: \"kubernetes.io/projected/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-kube-api-access-d76cm\") pod \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") "
Sep 04 22:09:04.087208 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087075    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbcq5\" (UniqueName: \"kubernetes.io/projected/84eecd63-5b01-4554-bad9-cc9f409f4232-kube-api-access-lbcq5\") pod \"84eecd63-5b01-4554-bad9-cc9f409f4232\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") "
Sep 04 22:09:04.087208 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087161    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3684bc49-1570-4032-9ff0-268bf457d691-plugins-config-volume\") pod \"3684bc49-1570-4032-9ff0-268bf457d691\" (UID: \"3684bc49-1570-4032-9ff0-268bf457d691\") "
Sep 04 22:09:04.087208 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087200    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-maas-api-tls\") pod \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") "
Sep 04 22:09:04.087472 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087238    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-openshift-service-ca\") pod \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") "
Sep 04 22:09:04.087472 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087260    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-metrics-certs\") pod \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\" (UID: \"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3\") "
Sep 04 22:09:04.087472 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087291    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84eecd63-5b01-4554-bad9-cc9f409f4232-plugins-config-volume\") pod \"84eecd63-5b01-4554-bad9-cc9f409f4232\" (UID: \"84eecd63-5b01-4554-bad9-cc9f409f4232\") "
Sep 04 22:09:04.087689 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.087656    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3684bc49-1570-4032-9ff0-268bf457d691-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3684bc49-1570-4032-9ff0-268bf457d691" (UID: "3684bc49-1570-4032-9ff0-268bf457d691"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:09:04.088115 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.088081    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" (UID: "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:09:04.088212 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.088124    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84eecd63-5b01-4554-bad9-cc9f409f4232-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84eecd63-5b01-4554-bad9-cc9f409f4232" (UID: "84eecd63-5b01-4554-bad9-cc9f409f4232"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:09:04.089591 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.089563    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-kube-api-access-d76cm" (OuterVolumeSpecName: "kube-api-access-d76cm") pod "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" (UID: "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3"). InnerVolumeSpecName "kube-api-access-d76cm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:04.089741 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.089716    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3684bc49-1570-4032-9ff0-268bf457d691-kube-api-access-nfhtq" (OuterVolumeSpecName: "kube-api-access-nfhtq") pod "3684bc49-1570-4032-9ff0-268bf457d691" (UID: "3684bc49-1570-4032-9ff0-268bf457d691"). InnerVolumeSpecName "kube-api-access-nfhtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:04.089741 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.089725    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84eecd63-5b01-4554-bad9-cc9f409f4232-kube-api-access-lbcq5" (OuterVolumeSpecName: "kube-api-access-lbcq5") pod "84eecd63-5b01-4554-bad9-cc9f409f4232" (UID: "84eecd63-5b01-4554-bad9-cc9f409f4232"). InnerVolumeSpecName "kube-api-access-lbcq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:04.090012 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.089992    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" (UID: "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:09:04.090072 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.090013    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" (UID: "bd34f2c5-b740-4708-9d07-03d0b8e6b2c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189044    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189070    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189080    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189088    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84eecd63-5b01-4554-bad9-cc9f409f4232-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189096    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfhtq\" (UniqueName: \"kubernetes.io/projected/3684bc49-1570-4032-9ff0-268bf457d691-kube-api-access-nfhtq\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189106    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d76cm\" (UniqueName: \"kubernetes.io/projected/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3-kube-api-access-d76cm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189114    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbcq5\" (UniqueName: \"kubernetes.io/projected/84eecd63-5b01-4554-bad9-cc9f409f4232-kube-api-access-lbcq5\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.189119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.189122    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3684bc49-1570-4032-9ff0-268bf457d691-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:04.620389 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.620359    2565 generic.go:356] "Generic (PLEG): container finished" podID="3684bc49-1570-4032-9ff0-268bf457d691" containerID="6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5" exitCode=0
Sep 04 22:09:04.620744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.620421    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"
Sep 04 22:09:04.620744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.620428    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2" event={"ID":"3684bc49-1570-4032-9ff0-268bf457d691","Type":"ContainerDied","Data":"6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5"}
Sep 04 22:09:04.620744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.620482    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2" event={"ID":"3684bc49-1570-4032-9ff0-268bf457d691","Type":"ContainerDied","Data":"ce479739821ad6a0c5c04c4c6afdec88ed2e7a157778112f6653f442b13654de"}
Sep 04 22:09:04.620744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.620511    2565 scope.go:117] "RemoveContainer" containerID="6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5"
Sep 04 22:09:04.621957 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.621931    2565 generic.go:356] "Generic (PLEG): container finished" podID="bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" containerID="a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a" exitCode=0
Sep 04 22:09:04.622082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.621995    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"
Sep 04 22:09:04.622498 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.622467    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8" event={"ID":"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3","Type":"ContainerDied","Data":"a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a"}
Sep 04 22:09:04.622657 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.622530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8" event={"ID":"bd34f2c5-b740-4708-9d07-03d0b8e6b2c3","Type":"ContainerDied","Data":"43b326de701694f00bc540e3a447b9789426284de200abed1097f62c486004f8"}
Sep 04 22:09:04.623838 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.623816    2565 generic.go:356] "Generic (PLEG): container finished" podID="84eecd63-5b01-4554-bad9-cc9f409f4232" containerID="1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434" exitCode=0
Sep 04 22:09:04.623966 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.623900    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"
Sep 04 22:09:04.624106 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.623900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p" event={"ID":"84eecd63-5b01-4554-bad9-cc9f409f4232","Type":"ContainerDied","Data":"1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434"}
Sep 04 22:09:04.624178 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.624129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p" event={"ID":"84eecd63-5b01-4554-bad9-cc9f409f4232","Type":"ContainerDied","Data":"08a0aeb1cf895742482cd2eab7e2282c035a7f75d41584f40c39d5bca6a427bb"}
Sep 04 22:09:04.630987 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.630841    2565 scope.go:117] "RemoveContainer" containerID="6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5"
Sep 04 22:09:04.631245 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:09:04.631220    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5\": container with ID starting with 6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5 not found: ID does not exist" containerID="6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5"
Sep 04 22:09:04.631311 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.631259    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5"} err="failed to get container status \"6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5\": rpc error: code = NotFound desc = could not find container \"6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5\": container with ID starting with 6facf986a7c326ffc3382aede7af464d22a752f2617b4722a8956466288d12a5 not found: ID does not exist"
Sep 04 22:09:04.631311 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.631282    2565 scope.go:117] "RemoveContainer" containerID="a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a"
Sep 04 22:09:04.640125 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.640108    2565 scope.go:117] "RemoveContainer" containerID="a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a"
Sep 04 22:09:04.640369 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:09:04.640328    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a\": container with ID starting with a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a not found: ID does not exist" containerID="a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a"
Sep 04 22:09:04.640421 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.640382    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a"} err="failed to get container status \"a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a\": rpc error: code = NotFound desc = could not find container \"a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a\": container with ID starting with a2f21ceda2ae11e367d49a2bb7867296895e7cf1a48d7cfdb60a5dc0e6aba25a not found: ID does not exist"
Sep 04 22:09:04.640421 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.640398    2565 scope.go:117] "RemoveContainer" containerID="1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434"
Sep 04 22:09:04.646714 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.646695    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"]
Sep 04 22:09:04.648763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.648742    2565 scope.go:117] "RemoveContainer" containerID="1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434"
Sep 04 22:09:04.649019 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:09:04.649004    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434\": container with ID starting with 1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434 not found: ID does not exist" containerID="1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434"
Sep 04 22:09:04.649081 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.649025    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434"} err="failed to get container status \"1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434\": rpc error: code = NotFound desc = could not find container \"1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434\": container with ID starting with 1ce705d91a249167dbb7edfa287fbae58f8b7af10bc18d5a0d8ba5e6fdfc6434 not found: ID does not exist"
Sep 04 22:09:04.650105 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.650088    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a5e1c2-668959c467-twtq2"]
Sep 04 22:09:04.659194 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.659174    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"]
Sep 04 22:09:04.663386 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.663370    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a5e1c2-7d84c588d6-nxr5p"]
Sep 04 22:09:04.673450 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.673432    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"]
Sep 04 22:09:04.677426 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:04.677407    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a5e1c2-fb69458d8-v6cx8"]
Sep 04 22:09:05.391203 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:05.391172    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3684bc49-1570-4032-9ff0-268bf457d691" path="/var/lib/kubelet/pods/3684bc49-1570-4032-9ff0-268bf457d691/volumes"
Sep 04 22:09:05.391565 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:05.391551    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84eecd63-5b01-4554-bad9-cc9f409f4232" path="/var/lib/kubelet/pods/84eecd63-5b01-4554-bad9-cc9f409f4232/volumes"
Sep 04 22:09:05.391896 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:05.391884    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" path="/var/lib/kubelet/pods/bd34f2c5-b740-4708-9d07-03d0b8e6b2c3/volumes"
Sep 04 22:09:08.048868 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.048826    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29" podUID="b1a74a38-7500-4c68-805d-5eebe2aa9f92" containerName="main" containerID="cri-o://907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229" gracePeriod=45
Sep 04 22:09:08.298252 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.298228    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:09:08.425959 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.425930    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-dshm\") pod \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") "
Sep 04 22:09:08.426153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.425971    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-model-cache\") pod \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") "
Sep 04 22:09:08.426153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426021    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tmp-dir\") pod \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") "
Sep 04 22:09:08.426153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426040    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9gsz\" (UniqueName: \"kubernetes.io/projected/b1a74a38-7500-4c68-805d-5eebe2aa9f92-kube-api-access-n9gsz\") pod \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") "
Sep 04 22:09:08.426153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426072    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tls-certs\") pod \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") "
Sep 04 22:09:08.426153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426102    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-home\") pod \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\" (UID: \"b1a74a38-7500-4c68-805d-5eebe2aa9f92\") "
Sep 04 22:09:08.426454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426308    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-model-cache" (OuterVolumeSpecName: "model-cache") pod "b1a74a38-7500-4c68-805d-5eebe2aa9f92" (UID: "b1a74a38-7500-4c68-805d-5eebe2aa9f92"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:08.426454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426322    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b1a74a38-7500-4c68-805d-5eebe2aa9f92" (UID: "b1a74a38-7500-4c68-805d-5eebe2aa9f92"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:08.426529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.426493    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-home" (OuterVolumeSpecName: "home") pod "b1a74a38-7500-4c68-805d-5eebe2aa9f92" (UID: "b1a74a38-7500-4c68-805d-5eebe2aa9f92"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:08.428325 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.428299    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-dshm" (OuterVolumeSpecName: "dshm") pod "b1a74a38-7500-4c68-805d-5eebe2aa9f92" (UID: "b1a74a38-7500-4c68-805d-5eebe2aa9f92"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:08.428459 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.428323    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b1a74a38-7500-4c68-805d-5eebe2aa9f92" (UID: "b1a74a38-7500-4c68-805d-5eebe2aa9f92"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:09:08.428459 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.428401    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a74a38-7500-4c68-805d-5eebe2aa9f92-kube-api-access-n9gsz" (OuterVolumeSpecName: "kube-api-access-n9gsz") pod "b1a74a38-7500-4c68-805d-5eebe2aa9f92" (UID: "b1a74a38-7500-4c68-805d-5eebe2aa9f92"). InnerVolumeSpecName "kube-api-access-n9gsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:08.527588 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.527559    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.527588 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.527587    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.527748 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.527600    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.527748 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.527612    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.527748 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.527624    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a74a38-7500-4c68-805d-5eebe2aa9f92-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.527748 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.527635    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9gsz\" (UniqueName: \"kubernetes.io/projected/b1a74a38-7500-4c68-805d-5eebe2aa9f92-kube-api-access-n9gsz\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.647870 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.647828    2565 generic.go:356] "Generic (PLEG): container finished" podID="b1a74a38-7500-4c68-805d-5eebe2aa9f92" containerID="907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229" exitCode=0
Sep 04 22:09:08.648031 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.647878    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"
Sep 04 22:09:08.648031 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.647898    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29" event={"ID":"b1a74a38-7500-4c68-805d-5eebe2aa9f92","Type":"ContainerDied","Data":"907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229"}
Sep 04 22:09:08.648031 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.647932    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29" event={"ID":"b1a74a38-7500-4c68-805d-5eebe2aa9f92","Type":"ContainerDied","Data":"08c2a98a0b0148692a85a1ad75592b4f28f17317c494b34b1c493af36b0ec0c5"}
Sep 04 22:09:08.648031 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.647949    2565 scope.go:117] "RemoveContainer" containerID="907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229"
Sep 04 22:09:08.656870 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.656849    2565 scope.go:117] "RemoveContainer" containerID="907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229"
Sep 04 22:09:08.657133 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:09:08.657116    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229\": container with ID starting with 907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229 not found: ID does not exist" containerID="907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229"
Sep 04 22:09:08.657205 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.657138    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229"} err="failed to get container status \"907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229\": rpc error: code = NotFound desc = could not find container \"907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229\": container with ID starting with 907f9b953ea99822c3179ec6c585141e4210d0fef9fc7f5622a89e852133d229 not found: ID does not exist"
Sep 04 22:09:08.667332 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.667313    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"]
Sep 04 22:09:08.673773 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:08.673749    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d4ad2d/test-model-d4ad2d-kserve-764b5dbdf9-4cb29"]
Sep 04 22:09:09.391052 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:09.391018    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a74a38-7500-4c68-805d-5eebe2aa9f92" path="/var/lib/kubelet/pods/b1a74a38-7500-4c68-805d-5eebe2aa9f92/volumes"
Sep 04 22:09:21.396489 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.396458    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"]
Sep 04 22:09:21.396913 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.396722    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" podUID="13669269-3537-4266-9c94-2dbf0c372e25" containerName="maas-api" containerID="cri-o://22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618" gracePeriod=30
Sep 04 22:09:21.469589 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.469557    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"]
Sep 04 22:09:21.469832 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.469771    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr" podUID="e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" containerName="payload-processing" containerID="cri-o://711f41d635972090e4cee47a93ede589da39a251f23e24e0cb223e0755133a3a" gracePeriod=30
Sep 04 22:09:21.710365 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.710301    2565 generic.go:356] "Generic (PLEG): container finished" podID="e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" containerID="711f41d635972090e4cee47a93ede589da39a251f23e24e0cb223e0755133a3a" exitCode=0
Sep 04 22:09:21.710526 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.710371    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr" event={"ID":"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c","Type":"ContainerDied","Data":"711f41d635972090e4cee47a93ede589da39a251f23e24e0cb223e0755133a3a"}
Sep 04 22:09:21.710526 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.710426    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr" event={"ID":"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c","Type":"ContainerDied","Data":"4288eb33b2d4cc979d0281ffce8013a135c8e5a1f27bc619c417ce00d19dea5a"}
Sep 04 22:09:21.710526 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.710442    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4288eb33b2d4cc979d0281ffce8013a135c8e5a1f27bc619c417ce00d19dea5a"
Sep 04 22:09:21.743905 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.743881    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:09:21.745204 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.745168    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" podUID="13669269-3537-4266-9c94-2dbf0c372e25" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.38:8443/health\": dial tcp 10.133.0.38:8443: connect: connection refused"
Sep 04 22:09:21.844051 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.844028    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ftxs\" (UniqueName: \"kubernetes.io/projected/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-kube-api-access-8ftxs\") pod \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") "
Sep 04 22:09:21.844157 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.844061    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-plugins-config-volume\") pod \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\" (UID: \"e61a2dbf-630c-4ee6-b9ab-cb472b68e16c\") "
Sep 04 22:09:21.844480 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.844451    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" (UID: "e61a2dbf-630c-4ee6-b9ab-cb472b68e16c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:09:21.846146 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.846122    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-kube-api-access-8ftxs" (OuterVolumeSpecName: "kube-api-access-8ftxs") pod "e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" (UID: "e61a2dbf-630c-4ee6-b9ab-cb472b68e16c"). InnerVolumeSpecName "kube-api-access-8ftxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:21.853868 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.853850    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:09:21.945157 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13669269-3537-4266-9c94-2dbf0c372e25-openshift-service-ca\") pod \"13669269-3537-4266-9c94-2dbf0c372e25\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") "
Sep 04 22:09:21.945157 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945110    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-maas-api-tls\") pod \"13669269-3537-4266-9c94-2dbf0c372e25\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") "
Sep 04 22:09:21.945157 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945136    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcgrl\" (UniqueName: \"kubernetes.io/projected/13669269-3537-4266-9c94-2dbf0c372e25-kube-api-access-gcgrl\") pod \"13669269-3537-4266-9c94-2dbf0c372e25\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") "
Sep 04 22:09:21.945438 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945190    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-metrics-certs\") pod \"13669269-3537-4266-9c94-2dbf0c372e25\" (UID: \"13669269-3537-4266-9c94-2dbf0c372e25\") "
Sep 04 22:09:21.945489 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945448    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13669269-3537-4266-9c94-2dbf0c372e25-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13669269-3537-4266-9c94-2dbf0c372e25" (UID: "13669269-3537-4266-9c94-2dbf0c372e25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:09:21.945544 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945524    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13669269-3537-4266-9c94-2dbf0c372e25-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:21.945595 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945542    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ftxs\" (UniqueName: \"kubernetes.io/projected/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-kube-api-access-8ftxs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:21.945595 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.945558    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:21.947392 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.947360    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "13669269-3537-4266-9c94-2dbf0c372e25" (UID: "13669269-3537-4266-9c94-2dbf0c372e25"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:09:21.947480 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.947411    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13669269-3537-4266-9c94-2dbf0c372e25-kube-api-access-gcgrl" (OuterVolumeSpecName: "kube-api-access-gcgrl") pod "13669269-3537-4266-9c94-2dbf0c372e25" (UID: "13669269-3537-4266-9c94-2dbf0c372e25"). InnerVolumeSpecName "kube-api-access-gcgrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:21.947480 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:21.947419    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13669269-3537-4266-9c94-2dbf0c372e25" (UID: "13669269-3537-4266-9c94-2dbf0c372e25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:09:22.046651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.046611    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:22.046651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.046641    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcgrl\" (UniqueName: \"kubernetes.io/projected/13669269-3537-4266-9c94-2dbf0c372e25-kube-api-access-gcgrl\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:22.046651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.046653    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13669269-3537-4266-9c94-2dbf0c372e25-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:09:22.721383 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.718398    2565 generic.go:356] "Generic (PLEG): container finished" podID="13669269-3537-4266-9c94-2dbf0c372e25" containerID="22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618" exitCode=0
Sep 04 22:09:22.721383 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.718551    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"
Sep 04 22:09:22.727366 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.722003    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"
Sep 04 22:09:22.729948 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.727551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" event={"ID":"13669269-3537-4266-9c94-2dbf0c372e25","Type":"ContainerDied","Data":"22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618"}
Sep 04 22:09:22.729948 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.727617    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5" event={"ID":"13669269-3537-4266-9c94-2dbf0c372e25","Type":"ContainerDied","Data":"0ff4bc0ac42bfa54ed49e3f6298a81493cc145439d80e7a1d7cc10fee95b6eb8"}
Sep 04 22:09:22.729948 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.727648    2565 scope.go:117] "RemoveContainer" containerID="22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618"
Sep 04 22:09:22.744097 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.743698    2565 scope.go:117] "RemoveContainer" containerID="22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618"
Sep 04 22:09:22.744097 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:09:22.744005    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618\": container with ID starting with 22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618 not found: ID does not exist" containerID="22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618"
Sep 04 22:09:22.744097 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.744037    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618"} err="failed to get container status \"22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618\": rpc error: code = NotFound desc = could not find container \"22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618\": container with ID starting with 22e3bb11f8e8b3017c0f4f08e1f83875cc0d30a2b6c3dda1d23a4befbf9d7618 not found: ID does not exist"
Sep 04 22:09:22.761120 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.761095    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"]
Sep 04 22:09:22.766672 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.766650    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d4ad2d-c6c9459c5-b76c5"]
Sep 04 22:09:22.777837 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.777816    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"]
Sep 04 22:09:22.781846 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:22.781828    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d4ad2d-66db96c549-42sfr"]
Sep 04 22:09:23.390352 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:23.390307    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13669269-3537-4266-9c94-2dbf0c372e25" path="/var/lib/kubelet/pods/13669269-3537-4266-9c94-2dbf0c372e25/volumes"
Sep 04 22:09:23.390702 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:23.390687    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" path="/var/lib/kubelet/pods/e61a2dbf-630c-4ee6-b9ab-cb472b68e16c/volumes"
Sep 04 22:09:32.996143 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996100    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"]
Sep 04 22:09:32.996562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996526    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" containerName="maas-api"
Sep 04 22:09:32.996562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996539    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" containerName="maas-api"
Sep 04 22:09:32.996562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996549    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a74a38-7500-4c68-805d-5eebe2aa9f92" containerName="main"
Sep 04 22:09:32.996562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996555    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a74a38-7500-4c68-805d-5eebe2aa9f92" containerName="main"
Sep 04 22:09:32.996562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996562    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13669269-3537-4266-9c94-2dbf0c372e25" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996568    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="13669269-3537-4266-9c94-2dbf0c372e25" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996575    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a745779-df57-4a82-9b62-54feaaf3f2a4" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996580    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a745779-df57-4a82-9b62-54feaaf3f2a4" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996589    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3684bc49-1570-4032-9ff0-268bf457d691" containerName="payload-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996594    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3684bc49-1570-4032-9ff0-268bf457d691" containerName="payload-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996600    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" containerName="payload-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996605    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" containerName="payload-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996618    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84eecd63-5b01-4554-bad9-cc9f409f4232" containerName="payload-pre-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996623    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="84eecd63-5b01-4554-bad9-cc9f409f4232" containerName="payload-pre-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996677    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="84eecd63-5b01-4554-bad9-cc9f409f4232" containerName="payload-pre-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996684    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e61a2dbf-630c-4ee6-b9ab-cb472b68e16c" containerName="payload-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996690    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a745779-df57-4a82-9b62-54feaaf3f2a4" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996696    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd34f2c5-b740-4708-9d07-03d0b8e6b2c3" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996702    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3684bc49-1570-4032-9ff0-268bf457d691" containerName="payload-processing"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996707    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="13669269-3537-4266-9c94-2dbf0c372e25" containerName="maas-api"
Sep 04 22:09:32.996724 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.996711    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a74a38-7500-4c68-805d-5eebe2aa9f92" containerName="main"
Sep 04 22:09:32.999708 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:32.999692    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.002126 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.002104    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-6680c7-serving-cert\""
Sep 04 22:09:33.014620 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.014596    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"]
Sep 04 22:09:33.141711 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.141675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.141711 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.141714    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.141923 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.141780    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg7v4\" (UniqueName: \"kubernetes.io/projected/695961b3-e923-4370-b2ec-19f3f9621b7d-kube-api-access-lg7v4\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.141923 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.141851    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/695961b3-e923-4370-b2ec-19f3f9621b7d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.242441 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.242405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.242441 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.242443    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.242661 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.242473    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg7v4\" (UniqueName: \"kubernetes.io/projected/695961b3-e923-4370-b2ec-19f3f9621b7d-kube-api-access-lg7v4\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.242661 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.242513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/695961b3-e923-4370-b2ec-19f3f9621b7d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.243196 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.243173    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/695961b3-e923-4370-b2ec-19f3f9621b7d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.244980 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.244950    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.245101 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.245010    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.251076 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.251020    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg7v4\" (UniqueName: \"kubernetes.io/projected/695961b3-e923-4370-b2ec-19f3f9621b7d-kube-api-access-lg7v4\") pod \"maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.310384 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.310334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:33.580579 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.580558    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"]
Sep 04 22:09:33.582970 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:09:33.582938    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod695961b3_e923_4370_b2ec_19f3f9621b7d.slice/crio-e12b87f92a8a61b411fa15aa2429d3dc0e7385b5113c56f84c89404a177d10c1 WatchSource:0}: Error finding container e12b87f92a8a61b411fa15aa2429d3dc0e7385b5113c56f84c89404a177d10c1: Status 404 returned error can't find the container with id e12b87f92a8a61b411fa15aa2429d3dc0e7385b5113c56f84c89404a177d10c1
Sep 04 22:09:33.628751 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.628718    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"]
Sep 04 22:09:33.633083 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.633067    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.635813 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.635792    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-6680c7-dockercfg-z9lvg\""
Sep 04 22:09:33.635917 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.635821    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-6680c7\""
Sep 04 22:09:33.645715 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.645692    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"]
Sep 04 22:09:33.670652 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.670626    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"]
Sep 04 22:09:33.673735 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.673720    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.683658 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.683634    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"]
Sep 04 22:09:33.747738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.747706    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn8ln\" (UniqueName: \"kubernetes.io/projected/05d94604-60e4-474c-9d27-5c9f2df1a216-kube-api-access-xn8ln\") pod \"payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.747738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.747742    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6td8\" (UniqueName: \"kubernetes.io/projected/edd2a55b-d9f2-46c1-9877-70fca4d71b60-kube-api-access-q6td8\") pod \"payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.747956 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.747768    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd2a55b-d9f2-46c1-9877-70fca4d71b60-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.747956 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.747894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05d94604-60e4-474c-9d27-5c9f2df1a216-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.772635 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.772602    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g" event={"ID":"695961b3-e923-4370-b2ec-19f3f9621b7d","Type":"ContainerStarted","Data":"e12b87f92a8a61b411fa15aa2429d3dc0e7385b5113c56f84c89404a177d10c1"}
Sep 04 22:09:33.849257 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.849178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd2a55b-d9f2-46c1-9877-70fca4d71b60-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.849257 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.849259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05d94604-60e4-474c-9d27-5c9f2df1a216-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.849475 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.849321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn8ln\" (UniqueName: \"kubernetes.io/projected/05d94604-60e4-474c-9d27-5c9f2df1a216-kube-api-access-xn8ln\") pod \"payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.849475 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.849374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6td8\" (UniqueName: \"kubernetes.io/projected/edd2a55b-d9f2-46c1-9877-70fca4d71b60-kube-api-access-q6td8\") pod \"payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.849945 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.849922    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05d94604-60e4-474c-9d27-5c9f2df1a216-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.850011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.849951    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd2a55b-d9f2-46c1-9877-70fca4d71b60-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.862738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.862704    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn8ln\" (UniqueName: \"kubernetes.io/projected/05d94604-60e4-474c-9d27-5c9f2df1a216-kube-api-access-xn8ln\") pod \"payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.874726 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.874696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6td8\" (UniqueName: \"kubernetes.io/projected/edd2a55b-d9f2-46c1-9877-70fca4d71b60-kube-api-access-q6td8\") pod \"payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:33.944152 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.944125    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:33.985413 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:33.985328    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:34.216667 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.216637    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"]
Sep 04 22:09:34.218539 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:09:34.218518    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d94604_60e4_474c_9d27_5c9f2df1a216.slice/crio-21bf580ebaf31065b9d18438163096d6d6cc98c816ec3e046889a11685f2c8c9 WatchSource:0}: Error finding container 21bf580ebaf31065b9d18438163096d6d6cc98c816ec3e046889a11685f2c8c9: Status 404 returned error can't find the container with id 21bf580ebaf31065b9d18438163096d6d6cc98c816ec3e046889a11685f2c8c9
Sep 04 22:09:34.272506 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.272476    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"]
Sep 04 22:09:34.285524 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:09:34.285497    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd2a55b_d9f2_46c1_9877_70fca4d71b60.slice/crio-83511280f531dbc116f2ddfb6cd8fd39f2b56f9bdd7d79cc3bfd0063aefb7302 WatchSource:0}: Error finding container 83511280f531dbc116f2ddfb6cd8fd39f2b56f9bdd7d79cc3bfd0063aefb7302: Status 404 returned error can't find the container with id 83511280f531dbc116f2ddfb6cd8fd39f2b56f9bdd7d79cc3bfd0063aefb7302
Sep 04 22:09:34.778642 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.778607    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g" event={"ID":"695961b3-e923-4370-b2ec-19f3f9621b7d","Type":"ContainerStarted","Data":"b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27"}
Sep 04 22:09:34.778825 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.778695    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:34.780217 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.780187    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9" event={"ID":"edd2a55b-d9f2-46c1-9877-70fca4d71b60","Type":"ContainerStarted","Data":"6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1"}
Sep 04 22:09:34.780349 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.780237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9" event={"ID":"edd2a55b-d9f2-46c1-9877-70fca4d71b60","Type":"ContainerStarted","Data":"83511280f531dbc116f2ddfb6cd8fd39f2b56f9bdd7d79cc3bfd0063aefb7302"}
Sep 04 22:09:34.780349 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.780267    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:34.781918 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.781894    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd" event={"ID":"05d94604-60e4-474c-9d27-5c9f2df1a216","Type":"ContainerStarted","Data":"a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2"}
Sep 04 22:09:34.782033 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.781975    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd" event={"ID":"05d94604-60e4-474c-9d27-5c9f2df1a216","Type":"ContainerStarted","Data":"21bf580ebaf31065b9d18438163096d6d6cc98c816ec3e046889a11685f2c8c9"}
Sep 04 22:09:34.782033 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.782004    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:34.799265 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.799217    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g" podStartSLOduration=2.358595258 podStartE2EDuration="2.799199472s" totalImagesPullingTime="440.604214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:09:34.797043247 +0000 UTC m=+949.881428093" watchObservedRunningTime="2026-09-04 22:09:34.799199472 +0000 UTC m=+949.883584316"
Sep 04 22:09:34.814169 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.814132    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9" podStartSLOduration=1.8141203849999998 podStartE2EDuration="1.814120385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:09:34.813455517 +0000 UTC m=+949.897840363" watchObservedRunningTime="2026-09-04 22:09:34.814120385 +0000 UTC m=+949.898505230"
Sep 04 22:09:34.829444 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:34.829407    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd" podStartSLOduration=1.829394732 podStartE2EDuration="1.829394732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:09:34.828456337 +0000 UTC m=+949.912841182" watchObservedRunningTime="2026-09-04 22:09:34.829394732 +0000 UTC m=+949.913779578"
Sep 04 22:09:40.793786 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:40.793752    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:09:45.790300 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:45.790214    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:09:45.790758 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:45.790302    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:09:55.455262 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.455230    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"]
Sep 04 22:09:55.460065 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.460047    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.462290 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.462264    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-806943a9-serving-cert\""
Sep 04 22:09:55.468961 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.468936    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"]
Sep 04 22:09:55.645788 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.645758    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1127f837-89c2-4129-a295-89817dd1688c-openshift-service-ca\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.645977 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.645795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-maas-api-tls\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.645977 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.645820    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-metrics-certs\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.645977 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.645889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqjlq\" (UniqueName: \"kubernetes.io/projected/1127f837-89c2-4129-a295-89817dd1688c-kube-api-access-jqjlq\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.746703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.746620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1127f837-89c2-4129-a295-89817dd1688c-openshift-service-ca\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.746703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.746658    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-maas-api-tls\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.746703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.746681    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-metrics-certs\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.746920 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.746731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqjlq\" (UniqueName: \"kubernetes.io/projected/1127f837-89c2-4129-a295-89817dd1688c-kube-api-access-jqjlq\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.747230 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.747208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1127f837-89c2-4129-a295-89817dd1688c-openshift-service-ca\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.749254 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.749234    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-maas-api-tls\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.749324 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.749280    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-metrics-certs\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.759459 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.759174    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqjlq\" (UniqueName: \"kubernetes.io/projected/1127f837-89c2-4129-a295-89817dd1688c-kube-api-access-jqjlq\") pod \"maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:55.771565 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:55.771540    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:56.044177 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:56.044149    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"]
Sep 04 22:09:56.046296 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:09:56.046270    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1127f837_89c2_4129_a295_89817dd1688c.slice/crio-09339bf29a88c9a0ec67f84ae70962a6c720d2a855648c71d47942e073a15907 WatchSource:0}: Error finding container 09339bf29a88c9a0ec67f84ae70962a6c720d2a855648c71d47942e073a15907: Status 404 returned error can't find the container with id 09339bf29a88c9a0ec67f84ae70962a6c720d2a855648c71d47942e073a15907
Sep 04 22:09:56.898473 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:56.898434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj" event={"ID":"1127f837-89c2-4129-a295-89817dd1688c","Type":"ContainerStarted","Data":"8b6a4f071a5f5bf856f3edce21610961dc96abfce70d1e96cb60d8e4190f337a"}
Sep 04 22:09:56.898882 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:56.898488    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj" event={"ID":"1127f837-89c2-4129-a295-89817dd1688c","Type":"ContainerStarted","Data":"09339bf29a88c9a0ec67f84ae70962a6c720d2a855648c71d47942e073a15907"}
Sep 04 22:09:56.898882 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:56.898533    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:09:56.917301 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:09:56.917261    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj" podStartSLOduration=1.526166618 podStartE2EDuration="1.91724745s" totalImagesPullingTime="391.080832ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:09:56.91658459 +0000 UTC m=+972.000969435" watchObservedRunningTime="2026-09-04 22:09:56.91724745 +0000 UTC m=+972.001632294"
Sep 04 22:10:02.328675 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.328640    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"]
Sep 04 22:10:02.332504 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.332470    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.335528 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.335506    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-7a26c9-serving-cert\""
Sep 04 22:10:02.343077 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.343053    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"]
Sep 04 22:10:02.407145 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.407108    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.407319 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.407160    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.407319 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.407206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmpc2\" (UniqueName: \"kubernetes.io/projected/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-kube-api-access-bmpc2\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.407319 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.407238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.508535 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.508494    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.508737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.508547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.508737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.508567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmpc2\" (UniqueName: \"kubernetes.io/projected/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-kube-api-access-bmpc2\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.508737 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.508586    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.508737 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:10:02.508685    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-serving-cert: secret "maas-api-e2e-shared-b-w3-7a26c9-serving-cert" not found
Sep 04 22:10:02.508929 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:10:02.508766    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls podName:ca41b5aa-95f8-45d6-9d36-c889ff9e49af nodeName:}" failed. No retries permitted until 2026-09-04 22:10:03.008747018 +0000 UTC m=+978.093131840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls") pod "maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" (UID: "ca41b5aa-95f8-45d6-9d36-c889ff9e49af") : secret "maas-api-e2e-shared-b-w3-7a26c9-serving-cert" not found
Sep 04 22:10:02.509662 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.509636    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.511407 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.511384    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.521759 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.521236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmpc2\" (UniqueName: \"kubernetes.io/projected/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-kube-api-access-bmpc2\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:02.909514 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:02.909482    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:10:03.014511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:03.014475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:03.017045 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:03.017015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:03.245869 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:03.245775    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:03.519325 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:03.519297    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"]
Sep 04 22:10:03.521288 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:10:03.521261    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca41b5aa_95f8_45d6_9d36_c889ff9e49af.slice/crio-252787ff7c8c093409edd34cdd48d849e05f6e3f95fadab2f9b14e17ab1d3360 WatchSource:0}: Error finding container 252787ff7c8c093409edd34cdd48d849e05f6e3f95fadab2f9b14e17ab1d3360: Status 404 returned error can't find the container with id 252787ff7c8c093409edd34cdd48d849e05f6e3f95fadab2f9b14e17ab1d3360
Sep 04 22:10:03.933654 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:03.933622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" event={"ID":"ca41b5aa-95f8-45d6-9d36-c889ff9e49af","Type":"ContainerStarted","Data":"252787ff7c8c093409edd34cdd48d849e05f6e3f95fadab2f9b14e17ab1d3360"}
Sep 04 22:10:04.940489 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:04.940446    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" event={"ID":"ca41b5aa-95f8-45d6-9d36-c889ff9e49af","Type":"ContainerStarted","Data":"b52fe622aa9e9bf171ca355109fdbfe1020a6c8c2f61a5e46ed70e4fc29d812f"}
Sep 04 22:10:04.940864 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:04.940584    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:04.958916 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:04.958869    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" podStartSLOduration=2.5868353060000002 podStartE2EDuration="2.9588546s" totalImagesPullingTime="372.019294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:10:04.957302634 +0000 UTC m=+980.041687478" watchObservedRunningTime="2026-09-04 22:10:04.9588546 +0000 UTC m=+980.043239443"
Sep 04 22:10:10.952249 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:10.952181    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:10:31.849671 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.849626    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"]
Sep 04 22:10:31.852939 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.852922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:31.855662 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.855637    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"kube-root-ca.crt\""
Sep 04 22:10:31.856289 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.856269    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"auth-test-model-6680c7-kserve-self-signed-certs\""
Sep 04 22:10:31.856383 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.856300    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"openshift-service-ca.crt\""
Sep 04 22:10:31.856383 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.856279    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"default-dockercfg-lxjlt\""
Sep 04 22:10:31.862393 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.862372    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"]
Sep 04 22:10:31.978759 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.978728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-dshm\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:31.978916 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.978774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-home\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:31.978916 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.978834    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-model-cache\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:31.978916 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.978910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f700175-2a1d-4a3a-9cf9-79950f06d999-tls-certs\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:31.979129 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.978942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-tmp-dir\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:31.979129 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:31.978973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dghj\" (UniqueName: \"kubernetes.io/projected/3f700175-2a1d-4a3a-9cf9-79950f06d999-kube-api-access-6dghj\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.079811 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.079778    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dghj\" (UniqueName: \"kubernetes.io/projected/3f700175-2a1d-4a3a-9cf9-79950f06d999-kube-api-access-6dghj\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.079996 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.079819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-dshm\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.079996 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.079866    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-home\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.079996 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.079922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-model-cache\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.079996 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.079971    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f700175-2a1d-4a3a-9cf9-79950f06d999-tls-certs\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.080215 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.080016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-tmp-dir\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.080433 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.080403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-home\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.080529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.080430    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-model-cache\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.080529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.080452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-tmp-dir\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.082267 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.082244    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-dshm\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.082621 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.082604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f700175-2a1d-4a3a-9cf9-79950f06d999-tls-certs\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.087716 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.087694    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dghj\" (UniqueName: \"kubernetes.io/projected/3f700175-2a1d-4a3a-9cf9-79950f06d999-kube-api-access-6dghj\") pod \"auth-test-model-6680c7-kserve-7595f7d795-gxzhz\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.162831 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.162750    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:32.435744 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:32.435723    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"]
Sep 04 22:10:32.437607 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:10:32.437576    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f700175_2a1d_4a3a_9cf9_79950f06d999.slice/crio-8d1f4a9182b58cb773b7627069dcc31ab0f0bf31e66d45e0dd30ac57f67c1622 WatchSource:0}: Error finding container 8d1f4a9182b58cb773b7627069dcc31ab0f0bf31e66d45e0dd30ac57f67c1622: Status 404 returned error can't find the container with id 8d1f4a9182b58cb773b7627069dcc31ab0f0bf31e66d45e0dd30ac57f67c1622
Sep 04 22:10:33.076979 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:33.076937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz" event={"ID":"3f700175-2a1d-4a3a-9cf9-79950f06d999","Type":"ContainerStarted","Data":"2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3"}
Sep 04 22:10:33.077372 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:33.076996    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz" event={"ID":"3f700175-2a1d-4a3a-9cf9-79950f06d999","Type":"ContainerStarted","Data":"8d1f4a9182b58cb773b7627069dcc31ab0f0bf31e66d45e0dd30ac57f67c1622"}
Sep 04 22:10:33.096511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:33.096462    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz" podStartSLOduration=2.096446041 podStartE2EDuration="2.096446041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:10:33.093996171 +0000 UTC m=+1008.178381016" watchObservedRunningTime="2026-09-04 22:10:33.096446041 +0000 UTC m=+1008.180830920"
Sep 04 22:10:36.973090 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:36.973056    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"]
Sep 04 22:10:36.973617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:36.973307    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj" podUID="1127f837-89c2-4129-a295-89817dd1688c" containerName="maas-api" containerID="cri-o://8b6a4f071a5f5bf856f3edce21610961dc96abfce70d1e96cb60d8e4190f337a" gracePeriod=30
Sep 04 22:10:37.099458 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.099426    2565 generic.go:356] "Generic (PLEG): container finished" podID="1127f837-89c2-4129-a295-89817dd1688c" containerID="8b6a4f071a5f5bf856f3edce21610961dc96abfce70d1e96cb60d8e4190f337a" exitCode=0
Sep 04 22:10:37.099583 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.099482    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj" event={"ID":"1127f837-89c2-4129-a295-89817dd1688c","Type":"ContainerDied","Data":"8b6a4f071a5f5bf856f3edce21610961dc96abfce70d1e96cb60d8e4190f337a"}
Sep 04 22:10:37.217148 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.217124    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:10:37.329521 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.329494    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1127f837-89c2-4129-a295-89817dd1688c-openshift-service-ca\") pod \"1127f837-89c2-4129-a295-89817dd1688c\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") "
Sep 04 22:10:37.329682 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.329590    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-metrics-certs\") pod \"1127f837-89c2-4129-a295-89817dd1688c\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") "
Sep 04 22:10:37.329682 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.329628    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-maas-api-tls\") pod \"1127f837-89c2-4129-a295-89817dd1688c\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") "
Sep 04 22:10:37.329682 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.329668    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqjlq\" (UniqueName: \"kubernetes.io/projected/1127f837-89c2-4129-a295-89817dd1688c-kube-api-access-jqjlq\") pod \"1127f837-89c2-4129-a295-89817dd1688c\" (UID: \"1127f837-89c2-4129-a295-89817dd1688c\") "
Sep 04 22:10:37.329926 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.329900    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1127f837-89c2-4129-a295-89817dd1688c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1127f837-89c2-4129-a295-89817dd1688c" (UID: "1127f837-89c2-4129-a295-89817dd1688c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:10:37.331882 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.331850    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1127f837-89c2-4129-a295-89817dd1688c" (UID: "1127f837-89c2-4129-a295-89817dd1688c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:10:37.331882 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.331858    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1127f837-89c2-4129-a295-89817dd1688c" (UID: "1127f837-89c2-4129-a295-89817dd1688c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:10:37.332050 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.331890    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1127f837-89c2-4129-a295-89817dd1688c-kube-api-access-jqjlq" (OuterVolumeSpecName: "kube-api-access-jqjlq") pod "1127f837-89c2-4129-a295-89817dd1688c" (UID: "1127f837-89c2-4129-a295-89817dd1688c"). InnerVolumeSpecName "kube-api-access-jqjlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:10:37.430688 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.430652    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.430688 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.430678    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqjlq\" (UniqueName: \"kubernetes.io/projected/1127f837-89c2-4129-a295-89817dd1688c-kube-api-access-jqjlq\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.430688 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.430689    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1127f837-89c2-4129-a295-89817dd1688c-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.430883 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:37.430698    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1127f837-89c2-4129-a295-89817dd1688c-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:10:38.105332 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:38.105298    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"
Sep 04 22:10:38.105801 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:38.105330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj" event={"ID":"1127f837-89c2-4129-a295-89817dd1688c","Type":"ContainerDied","Data":"09339bf29a88c9a0ec67f84ae70962a6c720d2a855648c71d47942e073a15907"}
Sep 04 22:10:38.105801 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:38.105407    2565 scope.go:117] "RemoveContainer" containerID="8b6a4f071a5f5bf856f3edce21610961dc96abfce70d1e96cb60d8e4190f337a"
Sep 04 22:10:38.126750 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:38.126724    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"]
Sep 04 22:10:38.131020 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:38.131001    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-806943a9-6996d8bdf5-8dvzj"]
Sep 04 22:10:39.390440 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:39.390409    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1127f837-89c2-4129-a295-89817dd1688c" path="/var/lib/kubelet/pods/1127f837-89c2-4129-a295-89817dd1688c/volumes"
Sep 04 22:10:42.163356 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:42.163316    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:42.163808 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:42.163404    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:42.176219 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:42.176193    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:10:43.141777 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:10:43.141746    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:11:02.542732 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.542694    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"]
Sep 04 22:11:02.543155 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.543071    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1127f837-89c2-4129-a295-89817dd1688c" containerName="maas-api"
Sep 04 22:11:02.543155 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.543083    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1127f837-89c2-4129-a295-89817dd1688c" containerName="maas-api"
Sep 04 22:11:02.543282 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.543162    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1127f837-89c2-4129-a295-89817dd1688c" containerName="maas-api"
Sep 04 22:11:02.547406 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.547382    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.549868 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.549844    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-fc3156-serving-cert\""
Sep 04 22:11:02.556734 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.556707    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"]
Sep 04 22:11:02.658847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.658812    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.659082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.658862    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.659082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.658955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.659082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.659000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbzfl\" (UniqueName: \"kubernetes.io/projected/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-kube-api-access-mbzfl\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.760352 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.760315    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.760539 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.760412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.760539 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.760491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.760539 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.760532    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbzfl\" (UniqueName: \"kubernetes.io/projected/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-kube-api-access-mbzfl\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.761207 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.761183    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.763013 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.762981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.763166 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.763142    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.768982 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.768961    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbzfl\" (UniqueName: \"kubernetes.io/projected/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-kube-api-access-mbzfl\") pod \"maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:02.859188 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:02.859151    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:03.133550 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:03.133515    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"]
Sep 04 22:11:03.136434 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:11:03.136398    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413a28b1_a348_40a2_a6dc_d5c2e27f6aac.slice/crio-a15eb53b00e3957266416cd993ff4e1e88cb5297879cc6f190e0f616b7945819 WatchSource:0}: Error finding container a15eb53b00e3957266416cd993ff4e1e88cb5297879cc6f190e0f616b7945819: Status 404 returned error can't find the container with id a15eb53b00e3957266416cd993ff4e1e88cb5297879cc6f190e0f616b7945819
Sep 04 22:11:03.137860 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:03.137840    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:11:03.230482 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:03.230416    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv" event={"ID":"413a28b1-a348-40a2-a6dc-d5c2e27f6aac","Type":"ContainerStarted","Data":"a15eb53b00e3957266416cd993ff4e1e88cb5297879cc6f190e0f616b7945819"}
Sep 04 22:11:04.236563 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:04.236524    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv" event={"ID":"413a28b1-a348-40a2-a6dc-d5c2e27f6aac","Type":"ContainerStarted","Data":"e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3"}
Sep 04 22:11:04.236936 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:04.236659    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:04.255716 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:04.255673    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv" podStartSLOduration=1.670204279 podStartE2EDuration="2.255661015s" totalImagesPullingTime="585.456736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:04.254413977 +0000 UTC m=+1039.338798822" watchObservedRunningTime="2026-09-04 22:11:04.255661015 +0000 UTC m=+1039.340045859"
Sep 04 22:11:10.035046 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.035013    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"]
Sep 04 22:11:10.038821 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.038797    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.042202 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.042177    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"default-dockercfg-vzzdw\""
Sep 04 22:11:10.042202 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.042192    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"kube-root-ca.crt\""
Sep 04 22:11:10.042202 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.042203    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"openshift-service-ca.crt\""
Sep 04 22:11:10.042473 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.042213    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"auth-test-model-7a26c9-kserve-self-signed-certs\""
Sep 04 22:11:10.050926 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.050891    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"]
Sep 04 22:11:10.124685 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.124643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-dshm\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.124685 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.124693    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4c2q\" (UniqueName: \"kubernetes.io/projected/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-kube-api-access-j4c2q\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.124917 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.124811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-model-cache\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.124917 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.124844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tls-certs\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.124996 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.124923    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-home\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.124996 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.124964    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tmp-dir\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226071 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4c2q\" (UniqueName: \"kubernetes.io/projected/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-kube-api-access-j4c2q\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-model-cache\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tls-certs\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226172    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-home\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226448 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226390    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tmp-dir\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226448 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226443    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-dshm\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226589 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-model-cache\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226589 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226582    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-home\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.226752 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.226720    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tmp-dir\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.228623 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.228602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-dshm\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.228782 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.228764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tls-certs\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.235115 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.235092    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4c2q\" (UniqueName: \"kubernetes.io/projected/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-kube-api-access-j4c2q\") pod \"auth-test-model-7a26c9-kserve-856b46dbf9-frgm5\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.245967 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.245947    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:11:10.351278 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.351249    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:10.623113 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:10.623090    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"]
Sep 04 22:11:10.625566 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:11:10.625533    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9420188_91d4_4fa9_bb03_a3d9d0516bb8.slice/crio-2e0caede678fc0219203b389ad86e7e15012a182ecddecfc6471301655de8a12 WatchSource:0}: Error finding container 2e0caede678fc0219203b389ad86e7e15012a182ecddecfc6471301655de8a12: Status 404 returned error can't find the container with id 2e0caede678fc0219203b389ad86e7e15012a182ecddecfc6471301655de8a12
Sep 04 22:11:11.271082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:11.271043    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5" event={"ID":"a9420188-91d4-4fa9-bb03-a3d9d0516bb8","Type":"ContainerStarted","Data":"b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f"}
Sep 04 22:11:11.271471 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:11.271095    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5" event={"ID":"a9420188-91d4-4fa9-bb03-a3d9d0516bb8","Type":"ContainerStarted","Data":"2e0caede678fc0219203b389ad86e7e15012a182ecddecfc6471301655de8a12"}
Sep 04 22:11:11.287931 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:11.287884    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5" podStartSLOduration=1.287862899 podStartE2EDuration="1.287862899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:11.286017682 +0000 UTC m=+1046.370402526" watchObservedRunningTime="2026-09-04 22:11:11.287862899 +0000 UTC m=+1046.372247743"
Sep 04 22:11:20.351496 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:20.351464    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:20.351932 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:20.351514    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:20.364206 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:20.364180    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:21.330465 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:21.330436    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:11:31.834910 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.834880    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"]
Sep 04 22:11:31.838650 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.838627    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:31.841918 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.841894    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-3abe72-serving-cert\""
Sep 04 22:11:31.849103 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.849079    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"]
Sep 04 22:11:31.928257 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.928216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l4c2\" (UniqueName: \"kubernetes.io/projected/164a6c3f-c745-4e39-a785-5f5f75f8844d-kube-api-access-4l4c2\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:31.928446 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.928283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:31.928446 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.928308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:31.928446 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:31.928328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164a6c3f-c745-4e39-a785-5f5f75f8844d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.029009 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.028975    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.029182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.029018    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.029182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.029035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164a6c3f-c745-4e39-a785-5f5f75f8844d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.029182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.029120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l4c2\" (UniqueName: \"kubernetes.io/projected/164a6c3f-c745-4e39-a785-5f5f75f8844d-kube-api-access-4l4c2\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.029835 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.029812    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164a6c3f-c745-4e39-a785-5f5f75f8844d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.031730 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.031704    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.031817 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.031732    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.037918 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.037898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l4c2\" (UniqueName: \"kubernetes.io/projected/164a6c3f-c745-4e39-a785-5f5f75f8844d-kube-api-access-4l4c2\") pod \"maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.153266 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.153182    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:32.434509 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:32.434483    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"]
Sep 04 22:11:32.436938 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:11:32.436901    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164a6c3f_c745_4e39_a785_5f5f75f8844d.slice/crio-9700a7101dbc48a3f1120a0de83dff8a3c222fc9e685f476fbb2d269c7439084 WatchSource:0}: Error finding container 9700a7101dbc48a3f1120a0de83dff8a3c222fc9e685f476fbb2d269c7439084: Status 404 returned error can't find the container with id 9700a7101dbc48a3f1120a0de83dff8a3c222fc9e685f476fbb2d269c7439084
Sep 04 22:11:33.379485 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:33.379447    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm" event={"ID":"164a6c3f-c745-4e39-a785-5f5f75f8844d","Type":"ContainerStarted","Data":"0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42"}
Sep 04 22:11:33.380009 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:33.379530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm" event={"ID":"164a6c3f-c745-4e39-a785-5f5f75f8844d","Type":"ContainerStarted","Data":"9700a7101dbc48a3f1120a0de83dff8a3c222fc9e685f476fbb2d269c7439084"}
Sep 04 22:11:33.380009 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:33.379648    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:33.398105 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:33.398061    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm" podStartSLOduration=2.088988994 podStartE2EDuration="2.398046636s" totalImagesPullingTime="309.057642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:33.395582061 +0000 UTC m=+1068.479966931" watchObservedRunningTime="2026-09-04 22:11:33.398046636 +0000 UTC m=+1068.482431480"
Sep 04 22:11:39.391055 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:39.390984    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:11:45.447095 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:11:45.447067    2565 scope.go:117] "RemoveContainer" containerID="20315eee46db88f4e3bf025c5065a0be3b1dca77335bea6a509c8c425259a73d"
Sep 04 22:12:13.680635 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.680596    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"]
Sep 04 22:12:13.684362 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.684312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.686803 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.686784    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bba28942-dockercfg-qjrnx\""
Sep 04 22:12:13.687147 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.687123    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bba28942\""
Sep 04 22:12:13.694722 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.694701    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"]
Sep 04 22:12:13.716083 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.716059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/867d66de-9bd9-42df-b682-5a867e6835a2-plugins-config-volume\") pod \"payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") " pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.716221 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.716200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrzs\" (UniqueName: \"kubernetes.io/projected/867d66de-9bd9-42df-b682-5a867e6835a2-kube-api-access-xvrzs\") pod \"payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") " pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.817247 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.817210    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrzs\" (UniqueName: \"kubernetes.io/projected/867d66de-9bd9-42df-b682-5a867e6835a2-kube-api-access-xvrzs\") pod \"payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") " pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.817442 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.817257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/867d66de-9bd9-42df-b682-5a867e6835a2-plugins-config-volume\") pod \"payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") " pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.817916 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.817897    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/867d66de-9bd9-42df-b682-5a867e6835a2-plugins-config-volume\") pod \"payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") " pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.825472 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.825437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrzs\" (UniqueName: \"kubernetes.io/projected/867d66de-9bd9-42df-b682-5a867e6835a2-kube-api-access-xvrzs\") pod \"payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") " pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:13.995762 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:13.995683    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:14.267542 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:14.267518    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"]
Sep 04 22:12:14.269771 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:12:14.269746    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod867d66de_9bd9_42df_b682_5a867e6835a2.slice/crio-d1715a999d1c0c8e40ca7a078cff46f1ebcf3610a110a0e7651ff9a83316260d WatchSource:0}: Error finding container d1715a999d1c0c8e40ca7a078cff46f1ebcf3610a110a0e7651ff9a83316260d: Status 404 returned error can't find the container with id d1715a999d1c0c8e40ca7a078cff46f1ebcf3610a110a0e7651ff9a83316260d
Sep 04 22:12:14.579084 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:14.579051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx" event={"ID":"867d66de-9bd9-42df-b682-5a867e6835a2","Type":"ContainerStarted","Data":"1e9f163c4ef2d660402a5df47050428c30c2221875bacf49e64d735681e512c4"}
Sep 04 22:12:14.579267 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:14.579114    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx" event={"ID":"867d66de-9bd9-42df-b682-5a867e6835a2","Type":"ContainerStarted","Data":"d1715a999d1c0c8e40ca7a078cff46f1ebcf3610a110a0e7651ff9a83316260d"}
Sep 04 22:12:14.579267 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:14.579141    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:14.598910 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:14.598864    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx" podStartSLOduration=1.5988472580000002 podStartE2EDuration="1.598847258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:12:14.596962099 +0000 UTC m=+1109.681346965" watchObservedRunningTime="2026-09-04 22:12:14.598847258 +0000 UTC m=+1109.683232102"
Sep 04 22:12:25.586502 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:25.586469    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:12:45.466683 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:12:45.466653    2565 scope.go:117] "RemoveContainer" containerID="711f41d635972090e4cee47a93ede589da39a251f23e24e0cb223e0755133a3a"
Sep 04 22:13:06.638894 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:06.638809    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"]
Sep 04 22:13:07.545712 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:07.545676    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"]
Sep 04 22:13:20.620932 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.620895    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"]
Sep 04 22:13:20.624647 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.624627    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.626978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.626957    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"sub-test-model-6680c7-kserve-self-signed-certs\""
Sep 04 22:13:20.632675 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.632597    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"]
Sep 04 22:13:20.755333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.755275    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-home\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.755333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.755371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-tmp-dir\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.755585 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.755393    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-dshm\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.755585 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.755429    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c81e24-dee8-4145-8026-2c03c060a4dc-tls-certs\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.755585 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.755555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6r7b\" (UniqueName: \"kubernetes.io/projected/19c81e24-dee8-4145-8026-2c03c060a4dc-kube-api-access-j6r7b\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.755688 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.755589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-model-cache\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856404 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-tmp-dir\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856404 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-dshm\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c81e24-dee8-4145-8026-2c03c060a4dc-tls-certs\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6r7b\" (UniqueName: \"kubernetes.io/projected/19c81e24-dee8-4145-8026-2c03c060a4dc-kube-api-access-j6r7b\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-model-cache\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856581    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-home\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856865 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856778    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-tmp-dir\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856906 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-home\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.856943 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.856901    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-model-cache\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.858932 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.858902    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-dshm\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.859076 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.859059    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c81e24-dee8-4145-8026-2c03c060a4dc-tls-certs\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.865309 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.865291    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6r7b\" (UniqueName: \"kubernetes.io/projected/19c81e24-dee8-4145-8026-2c03c060a4dc-kube-api-access-j6r7b\") pod \"sub-test-model-6680c7-kserve-76796d64c-45nzq\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:20.936140 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:20.936070    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:21.209914 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.209890    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"]
Sep 04 22:13:21.212208 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:13:21.212185    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19c81e24_dee8_4145_8026_2c03c060a4dc.slice/crio-1651dee7344f7e2bd65890ac8bdf73fb9db71c124b978666489ae61c8b06a710 WatchSource:0}: Error finding container 1651dee7344f7e2bd65890ac8bdf73fb9db71c124b978666489ae61c8b06a710: Status 404 returned error can't find the container with id 1651dee7344f7e2bd65890ac8bdf73fb9db71c124b978666489ae61c8b06a710
Sep 04 22:13:21.659418 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.659328    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz" podUID="3f700175-2a1d-4a3a-9cf9-79950f06d999" containerName="main" containerID="cri-o://2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3" gracePeriod=45
Sep 04 22:13:21.904393 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.904370    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:13:21.907056 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.907032    2565 generic.go:356] "Generic (PLEG): container finished" podID="3f700175-2a1d-4a3a-9cf9-79950f06d999" containerID="2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3" exitCode=0
Sep 04 22:13:21.907171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.907062    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"
Sep 04 22:13:21.907171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.907110    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz" event={"ID":"3f700175-2a1d-4a3a-9cf9-79950f06d999","Type":"ContainerDied","Data":"2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3"}
Sep 04 22:13:21.907171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.907149    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz" event={"ID":"3f700175-2a1d-4a3a-9cf9-79950f06d999","Type":"ContainerDied","Data":"8d1f4a9182b58cb773b7627069dcc31ab0f0bf31e66d45e0dd30ac57f67c1622"}
Sep 04 22:13:21.907171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.907165    2565 scope.go:117] "RemoveContainer" containerID="2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3"
Sep 04 22:13:21.908755 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.908736    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq" event={"ID":"19c81e24-dee8-4145-8026-2c03c060a4dc","Type":"ContainerStarted","Data":"e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d"}
Sep 04 22:13:21.908902 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.908764    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq" event={"ID":"19c81e24-dee8-4145-8026-2c03c060a4dc","Type":"ContainerStarted","Data":"1651dee7344f7e2bd65890ac8bdf73fb9db71c124b978666489ae61c8b06a710"}
Sep 04 22:13:21.915580 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.915562    2565 scope.go:117] "RemoveContainer" containerID="2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3"
Sep 04 22:13:21.915868 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:13:21.915850    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3\": container with ID starting with 2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3 not found: ID does not exist" containerID="2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3"
Sep 04 22:13:21.915909 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.915877    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3"} err="failed to get container status \"2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3\": rpc error: code = NotFound desc = could not find container \"2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3\": container with ID starting with 2288b4f1121795198c5a967014e1dc0d65ca8a84304cd26c1f7557714d8fb0a3 not found: ID does not exist"
Sep 04 22:13:21.944182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:21.944134    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq" podStartSLOduration=1.9441219589999998 podStartE2EDuration="1.944121959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:13:21.941444462 +0000 UTC m=+1177.025829321" watchObservedRunningTime="2026-09-04 22:13:21.944121959 +0000 UTC m=+1177.028506815"
Sep 04 22:13:22.067947 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.067918    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f700175-2a1d-4a3a-9cf9-79950f06d999-tls-certs\") pod \"3f700175-2a1d-4a3a-9cf9-79950f06d999\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") "
Sep 04 22:13:22.068142 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.067964    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-tmp-dir\") pod \"3f700175-2a1d-4a3a-9cf9-79950f06d999\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") "
Sep 04 22:13:22.068142 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068013    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-model-cache\") pod \"3f700175-2a1d-4a3a-9cf9-79950f06d999\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") "
Sep 04 22:13:22.068142 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068049    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-dshm\") pod \"3f700175-2a1d-4a3a-9cf9-79950f06d999\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") "
Sep 04 22:13:22.068142 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068081    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dghj\" (UniqueName: \"kubernetes.io/projected/3f700175-2a1d-4a3a-9cf9-79950f06d999-kube-api-access-6dghj\") pod \"3f700175-2a1d-4a3a-9cf9-79950f06d999\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") "
Sep 04 22:13:22.068142 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068128    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-home\") pod \"3f700175-2a1d-4a3a-9cf9-79950f06d999\" (UID: \"3f700175-2a1d-4a3a-9cf9-79950f06d999\") "
Sep 04 22:13:22.068419 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068266    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f700175-2a1d-4a3a-9cf9-79950f06d999" (UID: "3f700175-2a1d-4a3a-9cf9-79950f06d999"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.068419 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068279    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f700175-2a1d-4a3a-9cf9-79950f06d999" (UID: "3f700175-2a1d-4a3a-9cf9-79950f06d999"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.068512 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068473    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-home" (OuterVolumeSpecName: "home") pod "3f700175-2a1d-4a3a-9cf9-79950f06d999" (UID: "3f700175-2a1d-4a3a-9cf9-79950f06d999"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.068618 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068601    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.068685 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068624    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.068685 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.068638    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.070577 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.070546    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f700175-2a1d-4a3a-9cf9-79950f06d999-kube-api-access-6dghj" (OuterVolumeSpecName: "kube-api-access-6dghj") pod "3f700175-2a1d-4a3a-9cf9-79950f06d999" (UID: "3f700175-2a1d-4a3a-9cf9-79950f06d999"). InnerVolumeSpecName "kube-api-access-6dghj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:13:22.070762 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.070743    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-dshm" (OuterVolumeSpecName: "dshm") pod "3f700175-2a1d-4a3a-9cf9-79950f06d999" (UID: "3f700175-2a1d-4a3a-9cf9-79950f06d999"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.070830 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.070741    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f700175-2a1d-4a3a-9cf9-79950f06d999-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f700175-2a1d-4a3a-9cf9-79950f06d999" (UID: "3f700175-2a1d-4a3a-9cf9-79950f06d999"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:13:22.169152 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.169075    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f700175-2a1d-4a3a-9cf9-79950f06d999-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.169152 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.169101    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f700175-2a1d-4a3a-9cf9-79950f06d999-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.169152 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.169116    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dghj\" (UniqueName: \"kubernetes.io/projected/3f700175-2a1d-4a3a-9cf9-79950f06d999-kube-api-access-6dghj\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.230303 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.230270    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"]
Sep 04 22:13:22.234154 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.234129    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/auth-test-model-6680c7-kserve-7595f7d795-gxzhz"]
Sep 04 22:13:22.566286 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.566072    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5" podUID="a9420188-91d4-4fa9-bb03-a3d9d0516bb8" containerName="main" containerID="cri-o://b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f" gracePeriod=45
Sep 04 22:13:22.815968 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.815939    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:13:22.917686 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.917657    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9420188-91d4-4fa9-bb03-a3d9d0516bb8" containerID="b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f" exitCode=0
Sep 04 22:13:22.917847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.917698    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"
Sep 04 22:13:22.917847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.917735    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5" event={"ID":"a9420188-91d4-4fa9-bb03-a3d9d0516bb8","Type":"ContainerDied","Data":"b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f"}
Sep 04 22:13:22.917847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.917778    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5" event={"ID":"a9420188-91d4-4fa9-bb03-a3d9d0516bb8","Type":"ContainerDied","Data":"2e0caede678fc0219203b389ad86e7e15012a182ecddecfc6471301655de8a12"}
Sep 04 22:13:22.917847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.917796    2565 scope.go:117] "RemoveContainer" containerID="b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f"
Sep 04 22:13:22.926401 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.926385    2565 scope.go:117] "RemoveContainer" containerID="b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f"
Sep 04 22:13:22.926691 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:13:22.926673    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f\": container with ID starting with b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f not found: ID does not exist" containerID="b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f"
Sep 04 22:13:22.926749 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.926699    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f"} err="failed to get container status \"b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f\": rpc error: code = NotFound desc = could not find container \"b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f\": container with ID starting with b105a16c8c431e7b96b2429c97dcf42b889206087b6b77134ee7b20c551f617f not found: ID does not exist"
Sep 04 22:13:22.979088 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979059    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-home\") pod \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") "
Sep 04 22:13:22.979278 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979121    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-dshm\") pod \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") "
Sep 04 22:13:22.979278 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979150    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-model-cache\") pod \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") "
Sep 04 22:13:22.979278 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979248    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tls-certs\") pod \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") "
Sep 04 22:13:22.979463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979281    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4c2q\" (UniqueName: \"kubernetes.io/projected/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-kube-api-access-j4c2q\") pod \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") "
Sep 04 22:13:22.979463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979308    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-home" (OuterVolumeSpecName: "home") pod "a9420188-91d4-4fa9-bb03-a3d9d0516bb8" (UID: "a9420188-91d4-4fa9-bb03-a3d9d0516bb8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.979463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979366    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tmp-dir\") pod \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\" (UID: \"a9420188-91d4-4fa9-bb03-a3d9d0516bb8\") "
Sep 04 22:13:22.979621 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979493    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a9420188-91d4-4fa9-bb03-a3d9d0516bb8" (UID: "a9420188-91d4-4fa9-bb03-a3d9d0516bb8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.979725 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979700    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a9420188-91d4-4fa9-bb03-a3d9d0516bb8" (UID: "a9420188-91d4-4fa9-bb03-a3d9d0516bb8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.979799 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979723    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.979799 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.979743    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:22.981702 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.981674    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-kube-api-access-j4c2q" (OuterVolumeSpecName: "kube-api-access-j4c2q") pod "a9420188-91d4-4fa9-bb03-a3d9d0516bb8" (UID: "a9420188-91d4-4fa9-bb03-a3d9d0516bb8"). InnerVolumeSpecName "kube-api-access-j4c2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:13:22.981813 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.981728    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-dshm" (OuterVolumeSpecName: "dshm") pod "a9420188-91d4-4fa9-bb03-a3d9d0516bb8" (UID: "a9420188-91d4-4fa9-bb03-a3d9d0516bb8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:22.981857 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:22.981815    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a9420188-91d4-4fa9-bb03-a3d9d0516bb8" (UID: "a9420188-91d4-4fa9-bb03-a3d9d0516bb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:13:23.080649 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.080608    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:23.080649 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.080650    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4c2q\" (UniqueName: \"kubernetes.io/projected/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-kube-api-access-j4c2q\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:23.080872 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.080665    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:23.080872 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.080678    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9420188-91d4-4fa9-bb03-a3d9d0516bb8-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:23.241655 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.241622    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"]
Sep 04 22:13:23.246860 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.246832    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/auth-test-model-7a26c9-kserve-856b46dbf9-frgm5"]
Sep 04 22:13:23.391720 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.391691    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f700175-2a1d-4a3a-9cf9-79950f06d999" path="/var/lib/kubelet/pods/3f700175-2a1d-4a3a-9cf9-79950f06d999/volumes"
Sep 04 22:13:23.392164 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:23.392149    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9420188-91d4-4fa9-bb03-a3d9d0516bb8" path="/var/lib/kubelet/pods/a9420188-91d4-4fa9-bb03-a3d9d0516bb8/volumes"
Sep 04 22:13:26.759557 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:26.758940    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"]
Sep 04 22:13:26.759557 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:26.759205    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx" podUID="867d66de-9bd9-42df-b682-5a867e6835a2" containerName="payload-processing" containerID="cri-o://1e9f163c4ef2d660402a5df47050428c30c2221875bacf49e64d735681e512c4" gracePeriod=30
Sep 04 22:13:26.946077 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:26.946046    2565 generic.go:356] "Generic (PLEG): container finished" podID="867d66de-9bd9-42df-b682-5a867e6835a2" containerID="1e9f163c4ef2d660402a5df47050428c30c2221875bacf49e64d735681e512c4" exitCode=0
Sep 04 22:13:26.946248 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:26.946165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx" event={"ID":"867d66de-9bd9-42df-b682-5a867e6835a2","Type":"ContainerDied","Data":"1e9f163c4ef2d660402a5df47050428c30c2221875bacf49e64d735681e512c4"}
Sep 04 22:13:26.997606 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:26.997578    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:13:27.015218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.015147    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/867d66de-9bd9-42df-b682-5a867e6835a2-plugins-config-volume\") pod \"867d66de-9bd9-42df-b682-5a867e6835a2\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") "
Sep 04 22:13:27.015373 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.015236    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvrzs\" (UniqueName: \"kubernetes.io/projected/867d66de-9bd9-42df-b682-5a867e6835a2-kube-api-access-xvrzs\") pod \"867d66de-9bd9-42df-b682-5a867e6835a2\" (UID: \"867d66de-9bd9-42df-b682-5a867e6835a2\") "
Sep 04 22:13:27.015558 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.015517    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/867d66de-9bd9-42df-b682-5a867e6835a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "867d66de-9bd9-42df-b682-5a867e6835a2" (UID: "867d66de-9bd9-42df-b682-5a867e6835a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:13:27.017454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.017432    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/867d66de-9bd9-42df-b682-5a867e6835a2-kube-api-access-xvrzs" (OuterVolumeSpecName: "kube-api-access-xvrzs") pod "867d66de-9bd9-42df-b682-5a867e6835a2" (UID: "867d66de-9bd9-42df-b682-5a867e6835a2"). InnerVolumeSpecName "kube-api-access-xvrzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:13:27.116349 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.116299    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/867d66de-9bd9-42df-b682-5a867e6835a2-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:27.116509 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.116358    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvrzs\" (UniqueName: \"kubernetes.io/projected/867d66de-9bd9-42df-b682-5a867e6835a2-kube-api-access-xvrzs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:13:27.953136 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.953104    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"
Sep 04 22:13:27.953561 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.953145    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx" event={"ID":"867d66de-9bd9-42df-b682-5a867e6835a2","Type":"ContainerDied","Data":"d1715a999d1c0c8e40ca7a078cff46f1ebcf3610a110a0e7651ff9a83316260d"}
Sep 04 22:13:27.953561 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.953213    2565 scope.go:117] "RemoveContainer" containerID="1e9f163c4ef2d660402a5df47050428c30c2221875bacf49e64d735681e512c4"
Sep 04 22:13:27.969323 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.969299    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"]
Sep 04 22:13:27.972995 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:27.972971    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bba28942-76c6dcb87c-slbqx"]
Sep 04 22:13:29.390969 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:29.390935    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="867d66de-9bd9-42df-b682-5a867e6835a2" path="/var/lib/kubelet/pods/867d66de-9bd9-42df-b682-5a867e6835a2/volumes"
Sep 04 22:13:30.937110 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:30.937072    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:30.937532 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:30.937129    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:30.949354 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:30.949317    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:30.980418 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:30.980389    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:13:46.338994 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.338952    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"]
Sep 04 22:13:46.339486 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339388    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9420188-91d4-4fa9-bb03-a3d9d0516bb8" containerName="main"
Sep 04 22:13:46.339486 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339400    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9420188-91d4-4fa9-bb03-a3d9d0516bb8" containerName="main"
Sep 04 22:13:46.339486 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339409    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f700175-2a1d-4a3a-9cf9-79950f06d999" containerName="main"
Sep 04 22:13:46.339486 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339414    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f700175-2a1d-4a3a-9cf9-79950f06d999" containerName="main"
Sep 04 22:13:46.339486 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339433    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="867d66de-9bd9-42df-b682-5a867e6835a2" containerName="payload-processing"
Sep 04 22:13:46.339486 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339439    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="867d66de-9bd9-42df-b682-5a867e6835a2" containerName="payload-processing"
Sep 04 22:13:46.339788 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339497    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9420188-91d4-4fa9-bb03-a3d9d0516bb8" containerName="main"
Sep 04 22:13:46.339788 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339505    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="867d66de-9bd9-42df-b682-5a867e6835a2" containerName="payload-processing"
Sep 04 22:13:46.339788 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.339513    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f700175-2a1d-4a3a-9cf9-79950f06d999" containerName="main"
Sep 04 22:13:46.344107 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.344085    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.347217 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.347194    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"sub-test-model-7a26c9-kserve-self-signed-certs\""
Sep 04 22:13:46.347357 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.347229    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"kube-root-ca.crt\""
Sep 04 22:13:46.347357 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.347194    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"default-dockercfg-vzzdw\""
Sep 04 22:13:46.347357 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.347204    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"openshift-service-ca.crt\""
Sep 04 22:13:46.352635 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.352612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"]
Sep 04 22:13:46.396333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.396306    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr25t\" (UniqueName: \"kubernetes.io/projected/c5da2324-b56f-4825-b8b8-d5b777353b8d-kube-api-access-xr25t\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.396333 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.396351    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-dshm\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.396529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.396410    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-home\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.396529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.396450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-model-cache\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.396529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.396496    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-tmp-dir\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.396529 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.396518    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5da2324-b56f-4825-b8b8-d5b777353b8d-tls-certs\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.497658 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.497624    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-dshm\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.497802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.497667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-home\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.497802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.497711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-model-cache\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.497802 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.497776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-tmp-dir\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.497983 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.497803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5da2324-b56f-4825-b8b8-d5b777353b8d-tls-certs\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.497983 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.497854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr25t\" (UniqueName: \"kubernetes.io/projected/c5da2324-b56f-4825-b8b8-d5b777353b8d-kube-api-access-xr25t\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.498149 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.498128    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-home\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.498211 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.498165    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-model-cache\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.498211 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.498203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-tmp-dir\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.499992 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.499969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-dshm\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.500251 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.500235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5da2324-b56f-4825-b8b8-d5b777353b8d-tls-certs\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.504489 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.504471    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr25t\" (UniqueName: \"kubernetes.io/projected/c5da2324-b56f-4825-b8b8-d5b777353b8d-kube-api-access-xr25t\") pod \"sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.654738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.654644    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:46.930219 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:46.930141    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"]
Sep 04 22:13:46.934266 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:13:46.934229    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5da2324_b56f_4825_b8b8_d5b777353b8d.slice/crio-1a8800540423bbfabdb476beba3e75b29dfbbef3a7c27cc0b69ae39dd448c679 WatchSource:0}: Error finding container 1a8800540423bbfabdb476beba3e75b29dfbbef3a7c27cc0b69ae39dd448c679: Status 404 returned error can't find the container with id 1a8800540423bbfabdb476beba3e75b29dfbbef3a7c27cc0b69ae39dd448c679
Sep 04 22:13:47.050841 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:47.050802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx" event={"ID":"c5da2324-b56f-4825-b8b8-d5b777353b8d","Type":"ContainerStarted","Data":"0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113"}
Sep 04 22:13:47.051008 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:47.050852    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx" event={"ID":"c5da2324-b56f-4825-b8b8-d5b777353b8d","Type":"ContainerStarted","Data":"1a8800540423bbfabdb476beba3e75b29dfbbef3a7c27cc0b69ae39dd448c679"}
Sep 04 22:13:47.070270 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:47.070218    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx" podStartSLOduration=1.070200637 podStartE2EDuration="1.070200637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:13:47.069961445 +0000 UTC m=+1202.154346291" watchObservedRunningTime="2026-09-04 22:13:47.070200637 +0000 UTC m=+1202.154585484"
Sep 04 22:13:56.654911 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:56.654879    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:56.655411 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:56.654946    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:56.667651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:56.667627    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:13:57.113528 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:13:57.113501    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:14:22.376798 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:22.376761    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"]
Sep 04 22:14:23.228901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:23.228867    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"]
Sep 04 22:14:37.399025 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.398921    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq" podUID="19c81e24-dee8-4145-8026-2c03c060a4dc" containerName="main" containerID="cri-o://e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d" gracePeriod=45
Sep 04 22:14:37.653851 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.653797    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:14:37.772629 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.772599    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-dshm\") pod \"19c81e24-dee8-4145-8026-2c03c060a4dc\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") "
Sep 04 22:14:37.772791 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.772660    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6r7b\" (UniqueName: \"kubernetes.io/projected/19c81e24-dee8-4145-8026-2c03c060a4dc-kube-api-access-j6r7b\") pod \"19c81e24-dee8-4145-8026-2c03c060a4dc\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") "
Sep 04 22:14:37.772791 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.772718    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-tmp-dir\") pod \"19c81e24-dee8-4145-8026-2c03c060a4dc\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") "
Sep 04 22:14:37.772791 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.772776    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-home\") pod \"19c81e24-dee8-4145-8026-2c03c060a4dc\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") "
Sep 04 22:14:37.772921 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.772793    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-model-cache\") pod \"19c81e24-dee8-4145-8026-2c03c060a4dc\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") "
Sep 04 22:14:37.772921 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.772819    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c81e24-dee8-4145-8026-2c03c060a4dc-tls-certs\") pod \"19c81e24-dee8-4145-8026-2c03c060a4dc\" (UID: \"19c81e24-dee8-4145-8026-2c03c060a4dc\") "
Sep 04 22:14:37.773087 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.773053    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "19c81e24-dee8-4145-8026-2c03c060a4dc" (UID: "19c81e24-dee8-4145-8026-2c03c060a4dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:37.773087 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.773078    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-home" (OuterVolumeSpecName: "home") pod "19c81e24-dee8-4145-8026-2c03c060a4dc" (UID: "19c81e24-dee8-4145-8026-2c03c060a4dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:37.773240 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.773068    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19c81e24-dee8-4145-8026-2c03c060a4dc" (UID: "19c81e24-dee8-4145-8026-2c03c060a4dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:37.774980 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.774949    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19c81e24-dee8-4145-8026-2c03c060a4dc-kube-api-access-j6r7b" (OuterVolumeSpecName: "kube-api-access-j6r7b") pod "19c81e24-dee8-4145-8026-2c03c060a4dc" (UID: "19c81e24-dee8-4145-8026-2c03c060a4dc"). InnerVolumeSpecName "kube-api-access-j6r7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:14:37.775081 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.774977    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-dshm" (OuterVolumeSpecName: "dshm") pod "19c81e24-dee8-4145-8026-2c03c060a4dc" (UID: "19c81e24-dee8-4145-8026-2c03c060a4dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:37.775081 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.774983    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19c81e24-dee8-4145-8026-2c03c060a4dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19c81e24-dee8-4145-8026-2c03c060a4dc" (UID: "19c81e24-dee8-4145-8026-2c03c060a4dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:14:37.874217 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.874186    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:37.874217 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.874215    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:37.874217 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.874225    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c81e24-dee8-4145-8026-2c03c060a4dc-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:37.874471 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.874233    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:37.874471 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.874243    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6r7b\" (UniqueName: \"kubernetes.io/projected/19c81e24-dee8-4145-8026-2c03c060a4dc-kube-api-access-j6r7b\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:37.874471 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:37.874250    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c81e24-dee8-4145-8026-2c03c060a4dc-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:38.248815 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.248753    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx" podUID="c5da2324-b56f-4825-b8b8-d5b777353b8d" containerName="main" containerID="cri-o://0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113" gracePeriod=45
Sep 04 22:14:38.298808 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.298776    2565 generic.go:356] "Generic (PLEG): container finished" podID="19c81e24-dee8-4145-8026-2c03c060a4dc" containerID="e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d" exitCode=0
Sep 04 22:14:38.298942 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.298821    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"
Sep 04 22:14:38.298942 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.298861    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq" event={"ID":"19c81e24-dee8-4145-8026-2c03c060a4dc","Type":"ContainerDied","Data":"e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d"}
Sep 04 22:14:38.298942 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.298908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq" event={"ID":"19c81e24-dee8-4145-8026-2c03c060a4dc","Type":"ContainerDied","Data":"1651dee7344f7e2bd65890ac8bdf73fb9db71c124b978666489ae61c8b06a710"}
Sep 04 22:14:38.298942 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.298926    2565 scope.go:117] "RemoveContainer" containerID="e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d"
Sep 04 22:14:38.327942 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.327921    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"]
Sep 04 22:14:38.332042 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.332020    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/sub-test-model-6680c7-kserve-76796d64c-45nzq"]
Sep 04 22:14:38.359353 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.359317    2565 scope.go:117] "RemoveContainer" containerID="e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d"
Sep 04 22:14:38.359689 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:14:38.359669    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d\": container with ID starting with e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d not found: ID does not exist" containerID="e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d"
Sep 04 22:14:38.359759 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.359698    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d"} err="failed to get container status \"e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d\": rpc error: code = NotFound desc = could not find container \"e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d\": container with ID starting with e03fa888530ff83f957271f7273af2f5a4d6db2591056dd6c0f1b83160c0534d not found: ID does not exist"
Sep 04 22:14:38.498011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.497987    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:14:38.580168 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580140    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-model-cache\") pod \"c5da2324-b56f-4825-b8b8-d5b777353b8d\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") "
Sep 04 22:14:38.580327 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580236    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-dshm\") pod \"c5da2324-b56f-4825-b8b8-d5b777353b8d\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") "
Sep 04 22:14:38.580327 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580289    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5da2324-b56f-4825-b8b8-d5b777353b8d-tls-certs\") pod \"c5da2324-b56f-4825-b8b8-d5b777353b8d\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") "
Sep 04 22:14:38.580454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580393    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr25t\" (UniqueName: \"kubernetes.io/projected/c5da2324-b56f-4825-b8b8-d5b777353b8d-kube-api-access-xr25t\") pod \"c5da2324-b56f-4825-b8b8-d5b777353b8d\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") "
Sep 04 22:14:38.580454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580417    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c5da2324-b56f-4825-b8b8-d5b777353b8d" (UID: "c5da2324-b56f-4825-b8b8-d5b777353b8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:38.580454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580432    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-home\") pod \"c5da2324-b56f-4825-b8b8-d5b777353b8d\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") "
Sep 04 22:14:38.580579 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580499    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-tmp-dir\") pod \"c5da2324-b56f-4825-b8b8-d5b777353b8d\" (UID: \"c5da2324-b56f-4825-b8b8-d5b777353b8d\") "
Sep 04 22:14:38.580665 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580645    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-home" (OuterVolumeSpecName: "home") pod "c5da2324-b56f-4825-b8b8-d5b777353b8d" (UID: "c5da2324-b56f-4825-b8b8-d5b777353b8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:38.580863 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580760    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c5da2324-b56f-4825-b8b8-d5b777353b8d" (UID: "c5da2324-b56f-4825-b8b8-d5b777353b8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:38.580863 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.580849    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:38.582690 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.582656    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5da2324-b56f-4825-b8b8-d5b777353b8d-kube-api-access-xr25t" (OuterVolumeSpecName: "kube-api-access-xr25t") pod "c5da2324-b56f-4825-b8b8-d5b777353b8d" (UID: "c5da2324-b56f-4825-b8b8-d5b777353b8d"). InnerVolumeSpecName "kube-api-access-xr25t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:14:38.583001 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.582981    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-dshm" (OuterVolumeSpecName: "dshm") pod "c5da2324-b56f-4825-b8b8-d5b777353b8d" (UID: "c5da2324-b56f-4825-b8b8-d5b777353b8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:14:38.583001 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.582993    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5da2324-b56f-4825-b8b8-d5b777353b8d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c5da2324-b56f-4825-b8b8-d5b777353b8d" (UID: "c5da2324-b56f-4825-b8b8-d5b777353b8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:14:38.682085 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.682055    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr25t\" (UniqueName: \"kubernetes.io/projected/c5da2324-b56f-4825-b8b8-d5b777353b8d-kube-api-access-xr25t\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:38.682085 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.682085    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:38.682285 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.682100    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:38.682285 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.682133    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5da2324-b56f-4825-b8b8-d5b777353b8d-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:38.682285 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:38.682146    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5da2324-b56f-4825-b8b8-d5b777353b8d-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:14:39.305678 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.305643    2565 generic.go:356] "Generic (PLEG): container finished" podID="c5da2324-b56f-4825-b8b8-d5b777353b8d" containerID="0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113" exitCode=0
Sep 04 22:14:39.305873 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.305685    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"
Sep 04 22:14:39.305873 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.305697    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx" event={"ID":"c5da2324-b56f-4825-b8b8-d5b777353b8d","Type":"ContainerDied","Data":"0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113"}
Sep 04 22:14:39.305873 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.305743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx" event={"ID":"c5da2324-b56f-4825-b8b8-d5b777353b8d","Type":"ContainerDied","Data":"1a8800540423bbfabdb476beba3e75b29dfbbef3a7c27cc0b69ae39dd448c679"}
Sep 04 22:14:39.305873 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.305765    2565 scope.go:117] "RemoveContainer" containerID="0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113"
Sep 04 22:14:39.315705 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.315691    2565 scope.go:117] "RemoveContainer" containerID="0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113"
Sep 04 22:14:39.316014 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:14:39.315992    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113\": container with ID starting with 0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113 not found: ID does not exist" containerID="0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113"
Sep 04 22:14:39.316105 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.316017    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113"} err="failed to get container status \"0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113\": rpc error: code = NotFound desc = could not find container \"0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113\": container with ID starting with 0f64cb0e9ed5f6385bb98f7e27d210b1953089f1f89182090c66529437713113 not found: ID does not exist"
Sep 04 22:14:39.341083 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.341058    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"]
Sep 04 22:14:39.353108 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.353087    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/sub-test-model-7a26c9-kserve-5dcd968c75-4cnwx"]
Sep 04 22:14:39.391170 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.391139    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19c81e24-dee8-4145-8026-2c03c060a4dc" path="/var/lib/kubelet/pods/19c81e24-dee8-4145-8026-2c03c060a4dc/volumes"
Sep 04 22:14:39.391645 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:39.391623    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5da2324-b56f-4825-b8b8-d5b777353b8d" path="/var/lib/kubelet/pods/c5da2324-b56f-4825-b8b8-d5b777353b8d/volumes"
Sep 04 22:14:54.685102 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685066    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"]
Sep 04 22:14:54.685562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685492    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19c81e24-dee8-4145-8026-2c03c060a4dc" containerName="main"
Sep 04 22:14:54.685562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685504    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="19c81e24-dee8-4145-8026-2c03c060a4dc" containerName="main"
Sep 04 22:14:54.685562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685528    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5da2324-b56f-4825-b8b8-d5b777353b8d" containerName="main"
Sep 04 22:14:54.685562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685534    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5da2324-b56f-4825-b8b8-d5b777353b8d" containerName="main"
Sep 04 22:14:54.685694 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685602    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5da2324-b56f-4825-b8b8-d5b777353b8d" containerName="main"
Sep 04 22:14:54.685694 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.685609    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="19c81e24-dee8-4145-8026-2c03c060a4dc" containerName="main"
Sep 04 22:14:54.690021 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.690005    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.693868 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.693846    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"default-dockercfg-vzzdw\""
Sep 04 22:14:54.694011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.693871    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"kube-root-ca.crt\""
Sep 04 22:14:54.694011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.693935    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"rate-test-model-7a26c9-kserve-self-signed-certs\""
Sep 04 22:14:54.694135 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.694120    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7a26c9\"/\"openshift-service-ca.crt\""
Sep 04 22:14:54.713394 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.713359    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"]
Sep 04 22:14:54.727860 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.727823    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-tmp-dir\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.727978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.727907    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-dshm\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.727978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.727955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-model-cache\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.728070 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.727993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-home\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.728070 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.728036    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kd4q\" (UniqueName: \"kubernetes.io/projected/c8d284cc-572b-411b-8891-b1659f80cc34-kube-api-access-2kd4q\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.728151 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.728115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8d284cc-572b-411b-8891-b1659f80cc34-tls-certs\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.828813 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.828780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-dshm\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.828974 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.828820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-model-cache\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.828974 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.828840    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-home\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.829106 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.828967    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kd4q\" (UniqueName: \"kubernetes.io/projected/c8d284cc-572b-411b-8891-b1659f80cc34-kube-api-access-2kd4q\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.829106 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.829071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8d284cc-572b-411b-8891-b1659f80cc34-tls-certs\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.829200 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.829183    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-tmp-dir\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.829284 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.829258    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-model-cache\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.829373 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.829306    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-home\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.829527 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.829507    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-tmp-dir\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.831196 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.831180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-dshm\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.831683 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.831664    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8d284cc-572b-411b-8891-b1659f80cc34-tls-certs\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.839465 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.839439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kd4q\" (UniqueName: \"kubernetes.io/projected/c8d284cc-572b-411b-8891-b1659f80cc34-kube-api-access-2kd4q\") pod \"rate-test-model-7a26c9-kserve-7f69448999-j9h6z\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") " pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:54.999124 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:54.999032    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:14:55.292578 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:55.286919    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"]
Sep 04 22:14:55.393089 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:55.393058    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z" event={"ID":"c8d284cc-572b-411b-8891-b1659f80cc34","Type":"ContainerStarted","Data":"0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14"}
Sep 04 22:14:55.393240 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:55.393105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z" event={"ID":"c8d284cc-572b-411b-8891-b1659f80cc34","Type":"ContainerStarted","Data":"e63f541d3dd7515b8daf0fd8b6e523c903176b65df1e804db2744a21d5623ea5"}
Sep 04 22:14:55.412274 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:14:55.412226    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z" podStartSLOduration=1.412209947 podStartE2EDuration="1.412209947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:14:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:14:55.410564704 +0000 UTC m=+1270.494949548" watchObservedRunningTime="2026-09-04 22:14:55.412209947 +0000 UTC m=+1270.496594791"
Sep 04 22:15:04.999931 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:04.999894    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:15:04.999931 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:04.999945    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:15:05.013003 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:05.012977    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:15:05.452301 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:05.452276    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:15:17.829991 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.829957    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"]
Sep 04 22:15:17.833513 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.833495    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:17.838858 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.838839    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-25d966\""
Sep 04 22:15:17.838962 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.838839    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-25d966-dockercfg-fxsmv\""
Sep 04 22:15:17.848905 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.848882    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"]
Sep 04 22:15:17.868251 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.868225    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"]
Sep 04 22:15:17.871579 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.871560    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:17.886918 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.886895    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"]
Sep 04 22:15:17.956911 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.956879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6cgq\" (UniqueName: \"kubernetes.io/projected/abf047ee-c71b-455f-9e28-d13826b30202-kube-api-access-m6cgq\") pod \"payload-processing-e2e-api-a-25d966-58586b8b7-4z22r\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") " pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:17.957084 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.956955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrszd\" (UniqueName: \"kubernetes.io/projected/d484f570-0e6a-4d97-927c-376c95495b39-kube-api-access-jrszd\") pod \"payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:17.957084 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.957030    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d484f570-0e6a-4d97-927c-376c95495b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:17.957084 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:17.957069    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf047ee-c71b-455f-9e28-d13826b30202-plugins-config-volume\") pod \"payload-processing-e2e-api-a-25d966-58586b8b7-4z22r\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") " pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:18.058426 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.058392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cgq\" (UniqueName: \"kubernetes.io/projected/abf047ee-c71b-455f-9e28-d13826b30202-kube-api-access-m6cgq\") pod \"payload-processing-e2e-api-a-25d966-58586b8b7-4z22r\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") " pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:18.058619 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.058462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrszd\" (UniqueName: \"kubernetes.io/projected/d484f570-0e6a-4d97-927c-376c95495b39-kube-api-access-jrszd\") pod \"payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:18.058619 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.058541    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d484f570-0e6a-4d97-927c-376c95495b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:18.058619 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.058581    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf047ee-c71b-455f-9e28-d13826b30202-plugins-config-volume\") pod \"payload-processing-e2e-api-a-25d966-58586b8b7-4z22r\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") " pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:18.059191 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.059171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d484f570-0e6a-4d97-927c-376c95495b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:18.059245 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.059217    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf047ee-c71b-455f-9e28-d13826b30202-plugins-config-volume\") pod \"payload-processing-e2e-api-a-25d966-58586b8b7-4z22r\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") " pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:18.066654 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.066629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrszd\" (UniqueName: \"kubernetes.io/projected/d484f570-0e6a-4d97-927c-376c95495b39-kube-api-access-jrszd\") pod \"payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:18.066798 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.066772    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6cgq\" (UniqueName: \"kubernetes.io/projected/abf047ee-c71b-455f-9e28-d13826b30202-kube-api-access-m6cgq\") pod \"payload-processing-e2e-api-a-25d966-58586b8b7-4z22r\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") " pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:18.143892 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.143804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:18.182100 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.182060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:18.418906 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.418877    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"]
Sep 04 22:15:18.421057 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:15:18.421028    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd484f570_0e6a_4d97_927c_376c95495b39.slice/crio-e164ca4a6424d5a11630ffd904ffa98822ea3903a64c593f2b113ea1948ed190 WatchSource:0}: Error finding container e164ca4a6424d5a11630ffd904ffa98822ea3903a64c593f2b113ea1948ed190: Status 404 returned error can't find the container with id e164ca4a6424d5a11630ffd904ffa98822ea3903a64c593f2b113ea1948ed190
Sep 04 22:15:18.461785 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.461739    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"]
Sep 04 22:15:18.467317 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:15:18.467294    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf047ee_c71b_455f_9e28_d13826b30202.slice/crio-a68489763357e06670ebe107f47687d27daa7486071f6cd000655cab3da86abd WatchSource:0}: Error finding container a68489763357e06670ebe107f47687d27daa7486071f6cd000655cab3da86abd: Status 404 returned error can't find the container with id a68489763357e06670ebe107f47687d27daa7486071f6cd000655cab3da86abd
Sep 04 22:15:18.507497 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.507469    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r" event={"ID":"abf047ee-c71b-455f-9e28-d13826b30202","Type":"ContainerStarted","Data":"a68489763357e06670ebe107f47687d27daa7486071f6cd000655cab3da86abd"}
Sep 04 22:15:18.509187 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.509155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62" event={"ID":"d484f570-0e6a-4d97-927c-376c95495b39","Type":"ContainerStarted","Data":"6bcf43e14b53ca5600738212cebc4844125de747e5710cc7c2c19738023d27ae"}
Sep 04 22:15:18.509307 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.509202    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62" event={"ID":"d484f570-0e6a-4d97-927c-376c95495b39","Type":"ContainerStarted","Data":"e164ca4a6424d5a11630ffd904ffa98822ea3903a64c593f2b113ea1948ed190"}
Sep 04 22:15:18.509307 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.509265    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:18.525502 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:18.525452    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62" podStartSLOduration=1.525432221 podStartE2EDuration="1.525432221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:15:18.524228579 +0000 UTC m=+1293.608613423" watchObservedRunningTime="2026-09-04 22:15:18.525432221 +0000 UTC m=+1293.609817072"
Sep 04 22:15:19.515673 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:19.515635    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r" event={"ID":"abf047ee-c71b-455f-9e28-d13826b30202","Type":"ContainerStarted","Data":"5df02776b2f1c7f287933690ac0a5e6efcac1372c536a5a27ff484cadc485ada"}
Sep 04 22:15:19.516088 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:19.515724    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:19.532061 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:19.532013    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r" podStartSLOduration=2.531998857 podStartE2EDuration="2.531998857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:15:19.530795189 +0000 UTC m=+1294.615180034" watchObservedRunningTime="2026-09-04 22:15:19.531998857 +0000 UTC m=+1294.616383721"
Sep 04 22:15:29.518177 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:29.518144    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:15:30.523194 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:30.523167    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:15:58.024261 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:15:58.024226    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"]
Sep 04 22:16:04.310487 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.310457    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"]
Sep 04 22:16:04.315428 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.315407    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.318701 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.318684    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-cc92e2-dockercfg-b85ht\""
Sep 04 22:16:04.319408 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.319389    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-cc92e2\""
Sep 04 22:16:04.336936 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.336912    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"]
Sep 04 22:16:04.381530 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.381496    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"]
Sep 04 22:16:04.385329 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.385300    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.396533 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.396511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrpbk\" (UniqueName: \"kubernetes.io/projected/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-kube-api-access-lrpbk\") pod \"payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.396656 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.396553    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-797tb\" (UniqueName: \"kubernetes.io/projected/16c758da-3dc9-43b0-bd49-b4d40bae332a-kube-api-access-797tb\") pod \"payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.396726 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.396681    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16c758da-3dc9-43b0-bd49-b4d40bae332a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.396790 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.396760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.409605 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.409583    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"]
Sep 04 22:16:04.497781 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.497751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16c758da-3dc9-43b0-bd49-b4d40bae332a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.497928 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.497813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.497928 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.497875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrpbk\" (UniqueName: \"kubernetes.io/projected/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-kube-api-access-lrpbk\") pod \"payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.498037 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.497930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-797tb\" (UniqueName: \"kubernetes.io/projected/16c758da-3dc9-43b0-bd49-b4d40bae332a-kube-api-access-797tb\") pod \"payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.498592 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.498572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16c758da-3dc9-43b0-bd49-b4d40bae332a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.498732 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.498709    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.519261 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.519226    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrpbk\" (UniqueName: \"kubernetes.io/projected/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-kube-api-access-lrpbk\") pod \"payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.521689 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.521660    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-797tb\" (UniqueName: \"kubernetes.io/projected/16c758da-3dc9-43b0-bd49-b4d40bae332a-kube-api-access-797tb\") pod \"payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.626572 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.626535    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:04.696242 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.695633    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:04.899866 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.899840    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"]
Sep 04 22:16:04.901437 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:16:04.901413    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c758da_3dc9_43b0_bd49_b4d40bae332a.slice/crio-543badaa877d75a9c36d80274e97d579f66b5beb8759690eab0f62a2934774f7 WatchSource:0}: Error finding container 543badaa877d75a9c36d80274e97d579f66b5beb8759690eab0f62a2934774f7: Status 404 returned error can't find the container with id 543badaa877d75a9c36d80274e97d579f66b5beb8759690eab0f62a2934774f7
Sep 04 22:16:04.969605 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:04.969582    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"]
Sep 04 22:16:04.971034 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:16:04.971007    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f160840_b2bd_4e7d_b3e6_c1c22890f3ff.slice/crio-03339fe141f760185f58457cb8b99c7bc96f2167ad9e0d4e02579ec9dd1d88b8 WatchSource:0}: Error finding container 03339fe141f760185f58457cb8b99c7bc96f2167ad9e0d4e02579ec9dd1d88b8: Status 404 returned error can't find the container with id 03339fe141f760185f58457cb8b99c7bc96f2167ad9e0d4e02579ec9dd1d88b8
Sep 04 22:16:05.759043 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.759008    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q" event={"ID":"16c758da-3dc9-43b0-bd49-b4d40bae332a","Type":"ContainerStarted","Data":"0b28d6c1e62eee300692321e0684dff1a1e3b89407825f4f279bcea4478d45ec"}
Sep 04 22:16:05.759517 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.759059    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q" event={"ID":"16c758da-3dc9-43b0-bd49-b4d40bae332a","Type":"ContainerStarted","Data":"543badaa877d75a9c36d80274e97d579f66b5beb8759690eab0f62a2934774f7"}
Sep 04 22:16:05.759517 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.759087    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:05.760711 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.760675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg" event={"ID":"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff","Type":"ContainerStarted","Data":"4ea1e23532dd596aea0e0d5188e81c3dc57afdf0a1f6e90ca365562aa0fff62c"}
Sep 04 22:16:05.760835 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.760721    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg" event={"ID":"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff","Type":"ContainerStarted","Data":"03339fe141f760185f58457cb8b99c7bc96f2167ad9e0d4e02579ec9dd1d88b8"}
Sep 04 22:16:05.760835 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.760758    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:05.777244 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.777204    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q" podStartSLOduration=1.7771908 podStartE2EDuration="1.7771908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:05.774951442 +0000 UTC m=+1340.859336297" watchObservedRunningTime="2026-09-04 22:16:05.7771908 +0000 UTC m=+1340.861575645"
Sep 04 22:16:05.791198 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:05.791159    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg" podStartSLOduration=1.791146788 podStartE2EDuration="1.791146788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:05.789569056 +0000 UTC m=+1340.873953898" watchObservedRunningTime="2026-09-04 22:16:05.791146788 +0000 UTC m=+1340.875531631"
Sep 04 22:16:13.044494 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.044401    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z" podUID="c8d284cc-572b-411b-8891-b1659f80cc34" containerName="main" containerID="cri-o://0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14" gracePeriod=45
Sep 04 22:16:13.297880 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.297819    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:16:13.383225 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383188    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-model-cache\") pod \"c8d284cc-572b-411b-8891-b1659f80cc34\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") "
Sep 04 22:16:13.383225 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383226    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-dshm\") pod \"c8d284cc-572b-411b-8891-b1659f80cc34\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") "
Sep 04 22:16:13.383511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383251    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8d284cc-572b-411b-8891-b1659f80cc34-tls-certs\") pod \"c8d284cc-572b-411b-8891-b1659f80cc34\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") "
Sep 04 22:16:13.383511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383269    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-tmp-dir\") pod \"c8d284cc-572b-411b-8891-b1659f80cc34\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") "
Sep 04 22:16:13.383511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383320    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kd4q\" (UniqueName: \"kubernetes.io/projected/c8d284cc-572b-411b-8891-b1659f80cc34-kube-api-access-2kd4q\") pod \"c8d284cc-572b-411b-8891-b1659f80cc34\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") "
Sep 04 22:16:13.383511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383391    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-home\") pod \"c8d284cc-572b-411b-8891-b1659f80cc34\" (UID: \"c8d284cc-572b-411b-8891-b1659f80cc34\") "
Sep 04 22:16:13.383727 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383582    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-model-cache" (OuterVolumeSpecName: "model-cache") pod "c8d284cc-572b-411b-8891-b1659f80cc34" (UID: "c8d284cc-572b-411b-8891-b1659f80cc34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:13.383727 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383595    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c8d284cc-572b-411b-8891-b1659f80cc34" (UID: "c8d284cc-572b-411b-8891-b1659f80cc34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:13.383727 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383704    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:13.383727 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383721    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:13.383903 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.383756    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-home" (OuterVolumeSpecName: "home") pod "c8d284cc-572b-411b-8891-b1659f80cc34" (UID: "c8d284cc-572b-411b-8891-b1659f80cc34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:13.385563 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.385535    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-dshm" (OuterVolumeSpecName: "dshm") pod "c8d284cc-572b-411b-8891-b1659f80cc34" (UID: "c8d284cc-572b-411b-8891-b1659f80cc34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:13.385818 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.385796    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d284cc-572b-411b-8891-b1659f80cc34-kube-api-access-2kd4q" (OuterVolumeSpecName: "kube-api-access-2kd4q") pod "c8d284cc-572b-411b-8891-b1659f80cc34" (UID: "c8d284cc-572b-411b-8891-b1659f80cc34"). InnerVolumeSpecName "kube-api-access-2kd4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:16:13.386068 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.386049    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8d284cc-572b-411b-8891-b1659f80cc34-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c8d284cc-572b-411b-8891-b1659f80cc34" (UID: "c8d284cc-572b-411b-8891-b1659f80cc34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:16:13.485520 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.485479    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:13.485745 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.485731    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8d284cc-572b-411b-8891-b1659f80cc34-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:13.485845 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.485833    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8d284cc-572b-411b-8891-b1659f80cc34-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:13.485932 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.485921    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kd4q\" (UniqueName: \"kubernetes.io/projected/c8d284cc-572b-411b-8891-b1659f80cc34-kube-api-access-2kd4q\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:13.803515 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.803470    2565 generic.go:356] "Generic (PLEG): container finished" podID="c8d284cc-572b-411b-8891-b1659f80cc34" containerID="0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14" exitCode=0
Sep 04 22:16:13.803720 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.803525    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"
Sep 04 22:16:13.803720 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.803555    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z" event={"ID":"c8d284cc-572b-411b-8891-b1659f80cc34","Type":"ContainerDied","Data":"0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14"}
Sep 04 22:16:13.803720 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.803609    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z" event={"ID":"c8d284cc-572b-411b-8891-b1659f80cc34","Type":"ContainerDied","Data":"e63f541d3dd7515b8daf0fd8b6e523c903176b65df1e804db2744a21d5623ea5"}
Sep 04 22:16:13.803720 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.803632    2565 scope.go:117] "RemoveContainer" containerID="0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14"
Sep 04 22:16:13.814255 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.814230    2565 scope.go:117] "RemoveContainer" containerID="0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14"
Sep 04 22:16:13.814525 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:16:13.814503    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14\": container with ID starting with 0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14 not found: ID does not exist" containerID="0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14"
Sep 04 22:16:13.814611 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.814538    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14"} err="failed to get container status \"0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14\": rpc error: code = NotFound desc = could not find container \"0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14\": container with ID starting with 0f6d86f46eb81f71711e209a06cbb6268e1544b59d82493d22843fadae964c14 not found: ID does not exist"
Sep 04 22:16:13.821264 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.821242    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"]
Sep 04 22:16:13.823033 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:13.823015    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7a26c9/rate-test-model-7a26c9-kserve-7f69448999-j9h6z"]
Sep 04 22:16:15.390843 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:15.390808    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8d284cc-572b-411b-8891-b1659f80cc34" path="/var/lib/kubelet/pods/c8d284cc-572b-411b-8891-b1659f80cc34/volumes"
Sep 04 22:16:16.767952 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:16.767921    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:16:16.768374 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:16.768008    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:16:28.612775 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.612742    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"]
Sep 04 22:16:28.613162 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.612986    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r" podUID="abf047ee-c71b-455f-9e28-d13826b30202" containerName="payload-processing" containerID="cri-o://5df02776b2f1c7f287933690ac0a5e6efcac1372c536a5a27ff484cadc485ada" gracePeriod=30
Sep 04 22:16:28.621452 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.621194    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"]
Sep 04 22:16:28.621452 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.621441    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62" podUID="d484f570-0e6a-4d97-927c-376c95495b39" containerName="payload-pre-processing" containerID="cri-o://6bcf43e14b53ca5600738212cebc4844125de747e5710cc7c2c19738023d27ae" gracePeriod=30
Sep 04 22:16:28.881881 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.881848    2565 generic.go:356] "Generic (PLEG): container finished" podID="abf047ee-c71b-455f-9e28-d13826b30202" containerID="5df02776b2f1c7f287933690ac0a5e6efcac1372c536a5a27ff484cadc485ada" exitCode=0
Sep 04 22:16:28.882015 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.881924    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r" event={"ID":"abf047ee-c71b-455f-9e28-d13826b30202","Type":"ContainerDied","Data":"5df02776b2f1c7f287933690ac0a5e6efcac1372c536a5a27ff484cadc485ada"}
Sep 04 22:16:28.882015 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.881991    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r" event={"ID":"abf047ee-c71b-455f-9e28-d13826b30202","Type":"ContainerDied","Data":"a68489763357e06670ebe107f47687d27daa7486071f6cd000655cab3da86abd"}
Sep 04 22:16:28.882015 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.882013    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a68489763357e06670ebe107f47687d27daa7486071f6cd000655cab3da86abd"
Sep 04 22:16:28.883886 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.883868    2565 generic.go:356] "Generic (PLEG): container finished" podID="d484f570-0e6a-4d97-927c-376c95495b39" containerID="6bcf43e14b53ca5600738212cebc4844125de747e5710cc7c2c19738023d27ae" exitCode=0
Sep 04 22:16:28.883969 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.883899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62" event={"ID":"d484f570-0e6a-4d97-927c-376c95495b39","Type":"ContainerDied","Data":"6bcf43e14b53ca5600738212cebc4844125de747e5710cc7c2c19738023d27ae"}
Sep 04 22:16:28.898714 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.898697    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:16:28.901776 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:28.901758    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:16:29.037627 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.037594    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d484f570-0e6a-4d97-927c-376c95495b39-plugins-config-volume\") pod \"d484f570-0e6a-4d97-927c-376c95495b39\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") "
Sep 04 22:16:29.037816 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.037639    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf047ee-c71b-455f-9e28-d13826b30202-plugins-config-volume\") pod \"abf047ee-c71b-455f-9e28-d13826b30202\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") "
Sep 04 22:16:29.037816 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.037674    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6cgq\" (UniqueName: \"kubernetes.io/projected/abf047ee-c71b-455f-9e28-d13826b30202-kube-api-access-m6cgq\") pod \"abf047ee-c71b-455f-9e28-d13826b30202\" (UID: \"abf047ee-c71b-455f-9e28-d13826b30202\") "
Sep 04 22:16:29.037816 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.037729    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrszd\" (UniqueName: \"kubernetes.io/projected/d484f570-0e6a-4d97-927c-376c95495b39-kube-api-access-jrszd\") pod \"d484f570-0e6a-4d97-927c-376c95495b39\" (UID: \"d484f570-0e6a-4d97-927c-376c95495b39\") "
Sep 04 22:16:29.038054 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.038027    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf047ee-c71b-455f-9e28-d13826b30202-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abf047ee-c71b-455f-9e28-d13826b30202" (UID: "abf047ee-c71b-455f-9e28-d13826b30202"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:16:29.038109 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.038031    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d484f570-0e6a-4d97-927c-376c95495b39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d484f570-0e6a-4d97-927c-376c95495b39" (UID: "d484f570-0e6a-4d97-927c-376c95495b39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:16:29.039846 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.039811    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf047ee-c71b-455f-9e28-d13826b30202-kube-api-access-m6cgq" (OuterVolumeSpecName: "kube-api-access-m6cgq") pod "abf047ee-c71b-455f-9e28-d13826b30202" (UID: "abf047ee-c71b-455f-9e28-d13826b30202"). InnerVolumeSpecName "kube-api-access-m6cgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:16:29.039952 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.039844    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d484f570-0e6a-4d97-927c-376c95495b39-kube-api-access-jrszd" (OuterVolumeSpecName: "kube-api-access-jrszd") pod "d484f570-0e6a-4d97-927c-376c95495b39" (UID: "d484f570-0e6a-4d97-927c-376c95495b39"). InnerVolumeSpecName "kube-api-access-jrszd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:16:29.138848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.138776    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d484f570-0e6a-4d97-927c-376c95495b39-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:29.138848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.138800    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf047ee-c71b-455f-9e28-d13826b30202-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:29.138848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.138809    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6cgq\" (UniqueName: \"kubernetes.io/projected/abf047ee-c71b-455f-9e28-d13826b30202-kube-api-access-m6cgq\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:29.138848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.138818    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrszd\" (UniqueName: \"kubernetes.io/projected/d484f570-0e6a-4d97-927c-376c95495b39-kube-api-access-jrszd\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:16:29.890101 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.890066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62" event={"ID":"d484f570-0e6a-4d97-927c-376c95495b39","Type":"ContainerDied","Data":"e164ca4a6424d5a11630ffd904ffa98822ea3903a64c593f2b113ea1948ed190"}
Sep 04 22:16:29.890101 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.890088    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"
Sep 04 22:16:29.890613 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.890133    2565 scope.go:117] "RemoveContainer" containerID="6bcf43e14b53ca5600738212cebc4844125de747e5710cc7c2c19738023d27ae"
Sep 04 22:16:29.890613 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.890088    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"
Sep 04 22:16:29.909752 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.909727    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"]
Sep 04 22:16:29.913627 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.913605    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-25d966-58586b8b7-4z22r"]
Sep 04 22:16:29.922810 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.922791    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"]
Sep 04 22:16:29.926434 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:29.926415    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-25d966-78656d59d-6xt62"]
Sep 04 22:16:31.390599 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:31.390568    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf047ee-c71b-455f-9e28-d13826b30202" path="/var/lib/kubelet/pods/abf047ee-c71b-455f-9e28-d13826b30202/volumes"
Sep 04 22:16:31.390950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:31.390914    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d484f570-0e6a-4d97-927c-376c95495b39" path="/var/lib/kubelet/pods/d484f570-0e6a-4d97-927c-376c95495b39/volumes"
Sep 04 22:16:32.613468 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.613428    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"]
Sep 04 22:16:32.614102 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614073    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf047ee-c71b-455f-9e28-d13826b30202" containerName="payload-processing"
Sep 04 22:16:32.614102 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614100    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf047ee-c71b-455f-9e28-d13826b30202" containerName="payload-processing"
Sep 04 22:16:32.614239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614117    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d284cc-572b-411b-8891-b1659f80cc34" containerName="main"
Sep 04 22:16:32.614239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614126    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d284cc-572b-411b-8891-b1659f80cc34" containerName="main"
Sep 04 22:16:32.614239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614138    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d484f570-0e6a-4d97-927c-376c95495b39" containerName="payload-pre-processing"
Sep 04 22:16:32.614239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614147    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d484f570-0e6a-4d97-927c-376c95495b39" containerName="payload-pre-processing"
Sep 04 22:16:32.614459 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614290    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf047ee-c71b-455f-9e28-d13826b30202" containerName="payload-processing"
Sep 04 22:16:32.614459 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614305    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8d284cc-572b-411b-8891-b1659f80cc34" containerName="main"
Sep 04 22:16:32.614459 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.614317    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d484f570-0e6a-4d97-927c-376c95495b39" containerName="payload-pre-processing"
Sep 04 22:16:32.619901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.619879    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.622202 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.622178    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-f2fbdf-serving-cert\""
Sep 04 22:16:32.631047 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.631016    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"]
Sep 04 22:16:32.771786 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.771750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-metrics-certs\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.771992 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.771845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h777n\" (UniqueName: \"kubernetes.io/projected/ea70b92e-4b64-468e-867e-ebe7b1a418e0-kube-api-access-h777n\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.771992 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.771897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea70b92e-4b64-468e-867e-ebe7b1a418e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.771992 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.771956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.872901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.872823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-metrics-certs\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.872901 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.872877    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h777n\" (UniqueName: \"kubernetes.io/projected/ea70b92e-4b64-468e-867e-ebe7b1a418e0-kube-api-access-h777n\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.873126 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.872922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea70b92e-4b64-468e-867e-ebe7b1a418e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.873126 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.872943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.873667 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.873597    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea70b92e-4b64-468e-867e-ebe7b1a418e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.875451 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.875430    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.875612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.875592    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-metrics-certs\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.879785 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.879768    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h777n\" (UniqueName: \"kubernetes.io/projected/ea70b92e-4b64-468e-867e-ebe7b1a418e0-kube-api-access-h777n\") pod \"maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:32.933385 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:32.933329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:33.246910 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.246880    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"]
Sep 04 22:16:33.248958 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:16:33.248926    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea70b92e_4b64_468e_867e_ebe7b1a418e0.slice/crio-35a981aa92c3b76b4d4b68f2e62f967d6cf6c23f3912b858708ea235a123728d WatchSource:0}: Error finding container 35a981aa92c3b76b4d4b68f2e62f967d6cf6c23f3912b858708ea235a123728d: Status 404 returned error can't find the container with id 35a981aa92c3b76b4d4b68f2e62f967d6cf6c23f3912b858708ea235a123728d
Sep 04 22:16:33.250586 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.250565    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:16:33.348910 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.348874    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"]
Sep 04 22:16:33.353740 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.353706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.360674 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.360507    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f2fbdf\""
Sep 04 22:16:33.360674 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.360551    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f2fbdf-dockercfg-qbskl\""
Sep 04 22:16:33.376947 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.376914    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"]
Sep 04 22:16:33.443938 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.443907    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"]
Sep 04 22:16:33.447315 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.447300    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.457473 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.457450    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"]
Sep 04 22:16:33.478612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.478586    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a83595-5943-4f69-b2a9-3f7382fce96c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.478786 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.478666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmgvz\" (UniqueName: \"kubernetes.io/projected/79a83595-5943-4f69-b2a9-3f7382fce96c-kube-api-access-jmgvz\") pod \"payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.579890 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.579849    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec8298c2-9f11-4d09-b997-917d6bf3941e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.580054 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.579905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmgvz\" (UniqueName: \"kubernetes.io/projected/79a83595-5943-4f69-b2a9-3f7382fce96c-kube-api-access-jmgvz\") pod \"payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.580054 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.579968    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbmkt\" (UniqueName: \"kubernetes.io/projected/ec8298c2-9f11-4d09-b997-917d6bf3941e-kube-api-access-wbmkt\") pod \"payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.580176 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.580145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a83595-5943-4f69-b2a9-3f7382fce96c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.580815 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.580795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a83595-5943-4f69-b2a9-3f7382fce96c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.588769 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.588746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmgvz\" (UniqueName: \"kubernetes.io/projected/79a83595-5943-4f69-b2a9-3f7382fce96c-kube-api-access-jmgvz\") pod \"payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.664465 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.664417    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:33.680940 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.680908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec8298c2-9f11-4d09-b997-917d6bf3941e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.681050 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.680991    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbmkt\" (UniqueName: \"kubernetes.io/projected/ec8298c2-9f11-4d09-b997-917d6bf3941e-kube-api-access-wbmkt\") pod \"payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.681800 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.681776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec8298c2-9f11-4d09-b997-917d6bf3941e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.688509 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.688486    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbmkt\" (UniqueName: \"kubernetes.io/projected/ec8298c2-9f11-4d09-b997-917d6bf3941e-kube-api-access-wbmkt\") pod \"payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.758645 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.758620    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:33.912237 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.912157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6" event={"ID":"ea70b92e-4b64-468e-867e-ebe7b1a418e0","Type":"ContainerStarted","Data":"b9af8b13339ab56a4e4c4b8efd9ced2fec92df50aa218a23ba6c43a34dd6d8a1"}
Sep 04 22:16:33.912237 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.912213    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6" event={"ID":"ea70b92e-4b64-468e-867e-ebe7b1a418e0","Type":"ContainerStarted","Data":"35a981aa92c3b76b4d4b68f2e62f967d6cf6c23f3912b858708ea235a123728d"}
Sep 04 22:16:33.912430 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.912255    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:33.932600 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.932518    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6" podStartSLOduration=1.478135738 podStartE2EDuration="1.932503207s" totalImagesPullingTime="454.367469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:33.931965975 +0000 UTC m=+1369.016350810" watchObservedRunningTime="2026-09-04 22:16:33.932503207 +0000 UTC m=+1369.016888051"
Sep 04 22:16:33.948331 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:33.948305    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"]
Sep 04 22:16:33.950320 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:16:33.950300    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a83595_5943_4f69_b2a9_3f7382fce96c.slice/crio-ec75b6c43baa5ed64ee1e490cc1be3c81ea87da746f04b0cb5658601dd3b0130 WatchSource:0}: Error finding container ec75b6c43baa5ed64ee1e490cc1be3c81ea87da746f04b0cb5658601dd3b0130: Status 404 returned error can't find the container with id ec75b6c43baa5ed64ee1e490cc1be3c81ea87da746f04b0cb5658601dd3b0130
Sep 04 22:16:34.041685 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.041638    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"]
Sep 04 22:16:34.043891 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:16:34.043849    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8298c2_9f11_4d09_b997_917d6bf3941e.slice/crio-cc68f26284e476ac375f0d08b42581790b6eb0aa243c5f5c7cbcd7086e871890 WatchSource:0}: Error finding container cc68f26284e476ac375f0d08b42581790b6eb0aa243c5f5c7cbcd7086e871890: Status 404 returned error can't find the container with id cc68f26284e476ac375f0d08b42581790b6eb0aa243c5f5c7cbcd7086e871890
Sep 04 22:16:34.918583 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.918547    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv" event={"ID":"79a83595-5943-4f69-b2a9-3f7382fce96c","Type":"ContainerStarted","Data":"01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476"}
Sep 04 22:16:34.919025 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.918600    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv" event={"ID":"79a83595-5943-4f69-b2a9-3f7382fce96c","Type":"ContainerStarted","Data":"ec75b6c43baa5ed64ee1e490cc1be3c81ea87da746f04b0cb5658601dd3b0130"}
Sep 04 22:16:34.919025 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.918656    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:34.920395 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.920372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh" event={"ID":"ec8298c2-9f11-4d09-b997-917d6bf3941e","Type":"ContainerStarted","Data":"644323e61174430e960f5c482aa552022b72548b1936f6ade51764f7b630470f"}
Sep 04 22:16:34.920528 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.920409    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh" event={"ID":"ec8298c2-9f11-4d09-b997-917d6bf3941e","Type":"ContainerStarted","Data":"cc68f26284e476ac375f0d08b42581790b6eb0aa243c5f5c7cbcd7086e871890"}
Sep 04 22:16:34.920528 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.920447    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:16:34.933550 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.933500    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv" podStartSLOduration=1.933483878 podStartE2EDuration="1.933483878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:34.932958511 +0000 UTC m=+1370.017343380" watchObservedRunningTime="2026-09-04 22:16:34.933483878 +0000 UTC m=+1370.017868722"
Sep 04 22:16:34.951751 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:34.951705    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh" podStartSLOduration=1.951696237 podStartE2EDuration="1.951696237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:34.948874685 +0000 UTC m=+1370.033259529" watchObservedRunningTime="2026-09-04 22:16:34.951696237 +0000 UTC m=+1370.036081116"
Sep 04 22:16:39.925762 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:39.925732    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:16:45.928538 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:45.928500    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:16:45.928913 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:16:45.928582    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:17:01.686169 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:01.686123    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh_ec8298c2-9f11-4d09-b997-917d6bf3941e/payload-processing/0.log"
Sep 04 22:17:09.706659 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:09.706626    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"]
Sep 04 22:17:09.707111 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:09.706841    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv" podUID="413a28b1-a348-40a2-a6dc-d5c2e27f6aac" containerName="maas-api" containerID="cri-o://e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3" gracePeriod=30
Sep 04 22:17:09.964625 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:09.964570    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:17:10.110329 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.110289    2565 generic.go:356] "Generic (PLEG): container finished" podID="413a28b1-a348-40a2-a6dc-d5c2e27f6aac" containerID="e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3" exitCode=0
Sep 04 22:17:10.110329 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.110357    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"
Sep 04 22:17:10.110617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.110379    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv" event={"ID":"413a28b1-a348-40a2-a6dc-d5c2e27f6aac","Type":"ContainerDied","Data":"e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3"}
Sep 04 22:17:10.110617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.110436    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv" event={"ID":"413a28b1-a348-40a2-a6dc-d5c2e27f6aac","Type":"ContainerDied","Data":"a15eb53b00e3957266416cd993ff4e1e88cb5297879cc6f190e0f616b7945819"}
Sep 04 22:17:10.110617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.110457    2565 scope.go:117] "RemoveContainer" containerID="e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3"
Sep 04 22:17:10.119103 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.119075    2565 scope.go:117] "RemoveContainer" containerID="e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3"
Sep 04 22:17:10.119386 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:17:10.119367    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3\": container with ID starting with e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3 not found: ID does not exist" containerID="e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3"
Sep 04 22:17:10.119450 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.119398    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3"} err="failed to get container status \"e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3\": rpc error: code = NotFound desc = could not find container \"e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3\": container with ID starting with e2659a79af84bc070d768b717e459c3bb0f66fe8b18fa045954edf49b1af9ac3 not found: ID does not exist"
Sep 04 22:17:10.124718 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.124700    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-openshift-service-ca\") pod \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") "
Sep 04 22:17:10.124795 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.124731    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-maas-api-tls\") pod \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") "
Sep 04 22:17:10.124795 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.124749    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbzfl\" (UniqueName: \"kubernetes.io/projected/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-kube-api-access-mbzfl\") pod \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") "
Sep 04 22:17:10.124907 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.124878    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-metrics-certs\") pod \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\" (UID: \"413a28b1-a348-40a2-a6dc-d5c2e27f6aac\") "
Sep 04 22:17:10.125062 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.125031    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "413a28b1-a348-40a2-a6dc-d5c2e27f6aac" (UID: "413a28b1-a348-40a2-a6dc-d5c2e27f6aac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:10.125220 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.125205    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.126826 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.126801    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "413a28b1-a348-40a2-a6dc-d5c2e27f6aac" (UID: "413a28b1-a348-40a2-a6dc-d5c2e27f6aac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:17:10.126900 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.126846    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-kube-api-access-mbzfl" (OuterVolumeSpecName: "kube-api-access-mbzfl") pod "413a28b1-a348-40a2-a6dc-d5c2e27f6aac" (UID: "413a28b1-a348-40a2-a6dc-d5c2e27f6aac"). InnerVolumeSpecName "kube-api-access-mbzfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:17:10.126957 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.126934    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "413a28b1-a348-40a2-a6dc-d5c2e27f6aac" (UID: "413a28b1-a348-40a2-a6dc-d5c2e27f6aac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:17:10.226238 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.226168    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.226238 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.226193    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbzfl\" (UniqueName: \"kubernetes.io/projected/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-kube-api-access-mbzfl\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.226238 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.226202    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413a28b1-a348-40a2-a6dc-d5c2e27f6aac-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.431287 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.431257    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"]
Sep 04 22:17:10.435259 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:10.435224    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fc3156-748b9d5dfc-2tgkv"]
Sep 04 22:17:11.391231 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:11.391190    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="413a28b1-a348-40a2-a6dc-d5c2e27f6aac" path="/var/lib/kubelet/pods/413a28b1-a348-40a2-a6dc-d5c2e27f6aac/volumes"
Sep 04 22:17:27.153212 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.153175    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"]
Sep 04 22:17:27.153692 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.153442    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm" podUID="164a6c3f-c745-4e39-a785-5f5f75f8844d" containerName="maas-api" containerID="cri-o://0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42" gracePeriod=30
Sep 04 22:17:27.398842 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.398818    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:17:27.486063 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.485978    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l4c2\" (UniqueName: \"kubernetes.io/projected/164a6c3f-c745-4e39-a785-5f5f75f8844d-kube-api-access-4l4c2\") pod \"164a6c3f-c745-4e39-a785-5f5f75f8844d\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") "
Sep 04 22:17:27.486063 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.486052    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-metrics-certs\") pod \"164a6c3f-c745-4e39-a785-5f5f75f8844d\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") "
Sep 04 22:17:27.486295 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.486080    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-maas-api-tls\") pod \"164a6c3f-c745-4e39-a785-5f5f75f8844d\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") "
Sep 04 22:17:27.486295 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.486195    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164a6c3f-c745-4e39-a785-5f5f75f8844d-openshift-service-ca\") pod \"164a6c3f-c745-4e39-a785-5f5f75f8844d\" (UID: \"164a6c3f-c745-4e39-a785-5f5f75f8844d\") "
Sep 04 22:17:27.486636 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.486602    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/164a6c3f-c745-4e39-a785-5f5f75f8844d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "164a6c3f-c745-4e39-a785-5f5f75f8844d" (UID: "164a6c3f-c745-4e39-a785-5f5f75f8844d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:27.488430 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.488404    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "164a6c3f-c745-4e39-a785-5f5f75f8844d" (UID: "164a6c3f-c745-4e39-a785-5f5f75f8844d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:17:27.488430 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.488413    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164a6c3f-c745-4e39-a785-5f5f75f8844d-kube-api-access-4l4c2" (OuterVolumeSpecName: "kube-api-access-4l4c2") pod "164a6c3f-c745-4e39-a785-5f5f75f8844d" (UID: "164a6c3f-c745-4e39-a785-5f5f75f8844d"). InnerVolumeSpecName "kube-api-access-4l4c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:17:27.488574 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.488474    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "164a6c3f-c745-4e39-a785-5f5f75f8844d" (UID: "164a6c3f-c745-4e39-a785-5f5f75f8844d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:17:27.587122 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.587088    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164a6c3f-c745-4e39-a785-5f5f75f8844d-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.587122 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.587117    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l4c2\" (UniqueName: \"kubernetes.io/projected/164a6c3f-c745-4e39-a785-5f5f75f8844d-kube-api-access-4l4c2\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.587122 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.587129    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.587380 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:27.587137    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164a6c3f-c745-4e39-a785-5f5f75f8844d-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:17:28.202633 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.202599    2565 generic.go:356] "Generic (PLEG): container finished" podID="164a6c3f-c745-4e39-a785-5f5f75f8844d" containerID="0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42" exitCode=0
Sep 04 22:17:28.203091 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.202645    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"
Sep 04 22:17:28.203091 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.202692    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm" event={"ID":"164a6c3f-c745-4e39-a785-5f5f75f8844d","Type":"ContainerDied","Data":"0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42"}
Sep 04 22:17:28.203091 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.202742    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm" event={"ID":"164a6c3f-c745-4e39-a785-5f5f75f8844d","Type":"ContainerDied","Data":"9700a7101dbc48a3f1120a0de83dff8a3c222fc9e685f476fbb2d269c7439084"}
Sep 04 22:17:28.203091 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.202766    2565 scope.go:117] "RemoveContainer" containerID="0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42"
Sep 04 22:17:28.212255 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.212241    2565 scope.go:117] "RemoveContainer" containerID="0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42"
Sep 04 22:17:28.212542 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:17:28.212523    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42\": container with ID starting with 0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42 not found: ID does not exist" containerID="0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42"
Sep 04 22:17:28.212604 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.212551    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42"} err="failed to get container status \"0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42\": rpc error: code = NotFound desc = could not find container \"0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42\": container with ID starting with 0f6dcff000c846590890280b819400cb57aa88bab3fa608e20ea4ef660676f42 not found: ID does not exist"
Sep 04 22:17:28.223507 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.223487    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"]
Sep 04 22:17:28.227063 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:28.227045    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3abe72-6bffbcb4c8-wrccm"]
Sep 04 22:17:29.390050 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:29.390018    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="164a6c3f-c745-4e39-a785-5f5f75f8844d" path="/var/lib/kubelet/pods/164a6c3f-c745-4e39-a785-5f5f75f8844d/volumes"
Sep 04 22:17:32.112740 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:32.112698    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg_6f160840-b2bd-4e7d-b3e6-c1c22890f3ff/payload-processing/0.log"
Sep 04 22:17:32.347954 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:17:32.347921    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh_ec8298c2-9f11-4d09-b997-917d6bf3941e/payload-processing/0.log"
Sep 04 22:18:33.267651 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.267608    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"]
Sep 04 22:18:33.268201 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.267866    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg" podUID="6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" containerName="payload-processing" containerID="cri-o://4ea1e23532dd596aea0e0d5188e81c3dc57afdf0a1f6e90ca365562aa0fff62c" gracePeriod=30
Sep 04 22:18:33.278779 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.278170    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"]
Sep 04 22:18:33.278779 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.278420    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q" podUID="16c758da-3dc9-43b0-bd49-b4d40bae332a" containerName="payload-pre-processing" containerID="cri-o://0b28d6c1e62eee300692321e0684dff1a1e3b89407825f4f279bcea4478d45ec" gracePeriod=30
Sep 04 22:18:33.535045 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.535011    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" containerID="4ea1e23532dd596aea0e0d5188e81c3dc57afdf0a1f6e90ca365562aa0fff62c" exitCode=0
Sep 04 22:18:33.535176 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.535082    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg" event={"ID":"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff","Type":"ContainerDied","Data":"4ea1e23532dd596aea0e0d5188e81c3dc57afdf0a1f6e90ca365562aa0fff62c"}
Sep 04 22:18:33.536848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.536785    2565 generic.go:356] "Generic (PLEG): container finished" podID="16c758da-3dc9-43b0-bd49-b4d40bae332a" containerID="0b28d6c1e62eee300692321e0684dff1a1e3b89407825f4f279bcea4478d45ec" exitCode=0
Sep 04 22:18:33.536848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.536839    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q" event={"ID":"16c758da-3dc9-43b0-bd49-b4d40bae332a","Type":"ContainerDied","Data":"0b28d6c1e62eee300692321e0684dff1a1e3b89407825f4f279bcea4478d45ec"}
Sep 04 22:18:33.560103 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.560083    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:18:33.562974 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.562957    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:18:33.700521 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700491    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-plugins-config-volume\") pod \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") "
Sep 04 22:18:33.700697 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700588    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16c758da-3dc9-43b0-bd49-b4d40bae332a-plugins-config-volume\") pod \"16c758da-3dc9-43b0-bd49-b4d40bae332a\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") "
Sep 04 22:18:33.700697 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700614    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrpbk\" (UniqueName: \"kubernetes.io/projected/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-kube-api-access-lrpbk\") pod \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\" (UID: \"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff\") "
Sep 04 22:18:33.700697 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700656    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-797tb\" (UniqueName: \"kubernetes.io/projected/16c758da-3dc9-43b0-bd49-b4d40bae332a-kube-api-access-797tb\") pod \"16c758da-3dc9-43b0-bd49-b4d40bae332a\" (UID: \"16c758da-3dc9-43b0-bd49-b4d40bae332a\") "
Sep 04 22:18:33.700926 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700902    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" (UID: "6f160840-b2bd-4e7d-b3e6-c1c22890f3ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:18:33.700995 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700981    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:18:33.701041 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.700981    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16c758da-3dc9-43b0-bd49-b4d40bae332a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16c758da-3dc9-43b0-bd49-b4d40bae332a" (UID: "16c758da-3dc9-43b0-bd49-b4d40bae332a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:18:33.702851 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.702831    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-kube-api-access-lrpbk" (OuterVolumeSpecName: "kube-api-access-lrpbk") pod "6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" (UID: "6f160840-b2bd-4e7d-b3e6-c1c22890f3ff"). InnerVolumeSpecName "kube-api-access-lrpbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:18:33.702925 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.702908    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16c758da-3dc9-43b0-bd49-b4d40bae332a-kube-api-access-797tb" (OuterVolumeSpecName: "kube-api-access-797tb") pod "16c758da-3dc9-43b0-bd49-b4d40bae332a" (UID: "16c758da-3dc9-43b0-bd49-b4d40bae332a"). InnerVolumeSpecName "kube-api-access-797tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:18:33.801869 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.801794    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16c758da-3dc9-43b0-bd49-b4d40bae332a-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:18:33.801869 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.801821    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrpbk\" (UniqueName: \"kubernetes.io/projected/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff-kube-api-access-lrpbk\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:18:33.801869 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:33.801830    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-797tb\" (UniqueName: \"kubernetes.io/projected/16c758da-3dc9-43b0-bd49-b4d40bae332a-kube-api-access-797tb\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:18:34.543322 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.543228    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"
Sep 04 22:18:34.543322 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.543259    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg" event={"ID":"6f160840-b2bd-4e7d-b3e6-c1c22890f3ff","Type":"ContainerDied","Data":"03339fe141f760185f58457cb8b99c7bc96f2167ad9e0d4e02579ec9dd1d88b8"}
Sep 04 22:18:34.543872 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.543326    2565 scope.go:117] "RemoveContainer" containerID="4ea1e23532dd596aea0e0d5188e81c3dc57afdf0a1f6e90ca365562aa0fff62c"
Sep 04 22:18:34.544748 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.544730    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"
Sep 04 22:18:34.544809 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.544739    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q" event={"ID":"16c758da-3dc9-43b0-bd49-b4d40bae332a","Type":"ContainerDied","Data":"543badaa877d75a9c36d80274e97d579f66b5beb8759690eab0f62a2934774f7"}
Sep 04 22:18:34.552408 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.552392    2565 scope.go:117] "RemoveContainer" containerID="0b28d6c1e62eee300692321e0684dff1a1e3b89407825f4f279bcea4478d45ec"
Sep 04 22:18:34.568054 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.568034    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"]
Sep 04 22:18:34.573312 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.573291    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc92e2-5b86bb5f58-88w7q"]
Sep 04 22:18:34.582309 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.582287    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"]
Sep 04 22:18:34.585949 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:34.585930    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cc92e2-7f8969f67c-npblg"]
Sep 04 22:18:35.390692 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:35.390662    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16c758da-3dc9-43b0-bd49-b4d40bae332a" path="/var/lib/kubelet/pods/16c758da-3dc9-43b0-bd49-b4d40bae332a/volumes"
Sep 04 22:18:35.391002 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:18:35.390988    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" path="/var/lib/kubelet/pods/6f160840-b2bd-4e7d-b3e6-c1c22890f3ff/volumes"
Sep 04 22:19:32.639479 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.639444    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"]
Sep 04 22:19:32.640062 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.639703    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6" podUID="ea70b92e-4b64-468e-867e-ebe7b1a418e0" containerName="maas-api" containerID="cri-o://b9af8b13339ab56a4e4c4b8efd9ced2fec92df50aa218a23ba6c43a34dd6d8a1" gracePeriod=30
Sep 04 22:19:32.697608 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.697574    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"]
Sep 04 22:19:32.697843 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.697816    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh" podUID="ec8298c2-9f11-4d09-b997-917d6bf3941e" containerName="payload-processing" containerID="cri-o://644323e61174430e960f5c482aa552022b72548b1936f6ade51764f7b630470f" gracePeriod=30
Sep 04 22:19:32.705703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.705675    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"]
Sep 04 22:19:32.706409 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.705903    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv" podUID="79a83595-5943-4f69-b2a9-3f7382fce96c" containerName="payload-pre-processing" containerID="cri-o://01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476" gracePeriod=30
Sep 04 22:19:32.827194 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.827152    2565 generic.go:356] "Generic (PLEG): container finished" podID="ea70b92e-4b64-468e-867e-ebe7b1a418e0" containerID="b9af8b13339ab56a4e4c4b8efd9ced2fec92df50aa218a23ba6c43a34dd6d8a1" exitCode=0
Sep 04 22:19:32.827414 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.827231    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6" event={"ID":"ea70b92e-4b64-468e-867e-ebe7b1a418e0","Type":"ContainerDied","Data":"b9af8b13339ab56a4e4c4b8efd9ced2fec92df50aa218a23ba6c43a34dd6d8a1"}
Sep 04 22:19:32.829663 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.829636    2565 generic.go:356] "Generic (PLEG): container finished" podID="ec8298c2-9f11-4d09-b997-917d6bf3941e" containerID="644323e61174430e960f5c482aa552022b72548b1936f6ade51764f7b630470f" exitCode=0
Sep 04 22:19:32.829771 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.829680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh" event={"ID":"ec8298c2-9f11-4d09-b997-917d6bf3941e","Type":"ContainerDied","Data":"644323e61174430e960f5c482aa552022b72548b1936f6ade51764f7b630470f"}
Sep 04 22:19:32.979798 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.979765    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:19:32.996721 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:32.996695    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:19:33.008230 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.008209    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:19:33.053087 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053058    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec8298c2-9f11-4d09-b997-917d6bf3941e-plugins-config-volume\") pod \"ec8298c2-9f11-4d09-b997-917d6bf3941e\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") "
Sep 04 22:19:33.053220 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053096    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea70b92e-4b64-468e-867e-ebe7b1a418e0-openshift-service-ca\") pod \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") "
Sep 04 22:19:33.053220 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053136    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-maas-api-tls\") pod \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") "
Sep 04 22:19:33.053220 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053167    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h777n\" (UniqueName: \"kubernetes.io/projected/ea70b92e-4b64-468e-867e-ebe7b1a418e0-kube-api-access-h777n\") pod \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") "
Sep 04 22:19:33.053367 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053301    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmgvz\" (UniqueName: \"kubernetes.io/projected/79a83595-5943-4f69-b2a9-3f7382fce96c-kube-api-access-jmgvz\") pod \"79a83595-5943-4f69-b2a9-3f7382fce96c\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") "
Sep 04 22:19:33.053432 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053395    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbmkt\" (UniqueName: \"kubernetes.io/projected/ec8298c2-9f11-4d09-b997-917d6bf3941e-kube-api-access-wbmkt\") pod \"ec8298c2-9f11-4d09-b997-917d6bf3941e\" (UID: \"ec8298c2-9f11-4d09-b997-917d6bf3941e\") "
Sep 04 22:19:33.053494 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053458    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-metrics-certs\") pod \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\" (UID: \"ea70b92e-4b64-468e-867e-ebe7b1a418e0\") "
Sep 04 22:19:33.053494 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053487    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a83595-5943-4f69-b2a9-3f7382fce96c-plugins-config-volume\") pod \"79a83595-5943-4f69-b2a9-3f7382fce96c\" (UID: \"79a83595-5943-4f69-b2a9-3f7382fce96c\") "
Sep 04 22:19:33.053608 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053499    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec8298c2-9f11-4d09-b997-917d6bf3941e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec8298c2-9f11-4d09-b997-917d6bf3941e" (UID: "ec8298c2-9f11-4d09-b997-917d6bf3941e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:33.053608 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053549    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea70b92e-4b64-468e-867e-ebe7b1a418e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea70b92e-4b64-468e-867e-ebe7b1a418e0" (UID: "ea70b92e-4b64-468e-867e-ebe7b1a418e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:33.053812 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053793    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec8298c2-9f11-4d09-b997-917d6bf3941e-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.054060 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053816    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea70b92e-4b64-468e-867e-ebe7b1a418e0-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.054060 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.053971    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79a83595-5943-4f69-b2a9-3f7382fce96c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79a83595-5943-4f69-b2a9-3f7382fce96c" (UID: "79a83595-5943-4f69-b2a9-3f7382fce96c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:33.055462 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.055441    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea70b92e-4b64-468e-867e-ebe7b1a418e0-kube-api-access-h777n" (OuterVolumeSpecName: "kube-api-access-h777n") pod "ea70b92e-4b64-468e-867e-ebe7b1a418e0" (UID: "ea70b92e-4b64-468e-867e-ebe7b1a418e0"). InnerVolumeSpecName "kube-api-access-h777n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:33.056002 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.055982    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea70b92e-4b64-468e-867e-ebe7b1a418e0" (UID: "ea70b92e-4b64-468e-867e-ebe7b1a418e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:19:33.056078 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.055993    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a83595-5943-4f69-b2a9-3f7382fce96c-kube-api-access-jmgvz" (OuterVolumeSpecName: "kube-api-access-jmgvz") pod "79a83595-5943-4f69-b2a9-3f7382fce96c" (UID: "79a83595-5943-4f69-b2a9-3f7382fce96c"). InnerVolumeSpecName "kube-api-access-jmgvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:33.056078 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.056064    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec8298c2-9f11-4d09-b997-917d6bf3941e-kube-api-access-wbmkt" (OuterVolumeSpecName: "kube-api-access-wbmkt") pod "ec8298c2-9f11-4d09-b997-917d6bf3941e" (UID: "ec8298c2-9f11-4d09-b997-917d6bf3941e"). InnerVolumeSpecName "kube-api-access-wbmkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:33.056173 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.056159    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea70b92e-4b64-468e-867e-ebe7b1a418e0" (UID: "ea70b92e-4b64-468e-867e-ebe7b1a418e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:19:33.155446 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.155410    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.155676 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.155659    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79a83595-5943-4f69-b2a9-3f7382fce96c-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.155778 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.155766    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea70b92e-4b64-468e-867e-ebe7b1a418e0-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.155883 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.155872    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h777n\" (UniqueName: \"kubernetes.io/projected/ea70b92e-4b64-468e-867e-ebe7b1a418e0-kube-api-access-h777n\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.155959 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.155948    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmgvz\" (UniqueName: \"kubernetes.io/projected/79a83595-5943-4f69-b2a9-3f7382fce96c-kube-api-access-jmgvz\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.156035 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.156024    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbmkt\" (UniqueName: \"kubernetes.io/projected/ec8298c2-9f11-4d09-b997-917d6bf3941e-kube-api-access-wbmkt\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:33.836531 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.836491    2565 generic.go:356] "Generic (PLEG): container finished" podID="79a83595-5943-4f69-b2a9-3f7382fce96c" containerID="01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476" exitCode=0
Sep 04 22:19:33.836998 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.836535    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv" event={"ID":"79a83595-5943-4f69-b2a9-3f7382fce96c","Type":"ContainerDied","Data":"01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476"}
Sep 04 22:19:33.836998 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.836597    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv" event={"ID":"79a83595-5943-4f69-b2a9-3f7382fce96c","Type":"ContainerDied","Data":"ec75b6c43baa5ed64ee1e490cc1be3c81ea87da746f04b0cb5658601dd3b0130"}
Sep 04 22:19:33.836998 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.836553    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"
Sep 04 22:19:33.836998 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.836625    2565 scope.go:117] "RemoveContainer" containerID="01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476"
Sep 04 22:19:33.837978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.837963    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"
Sep 04 22:19:33.838070 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.837962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6" event={"ID":"ea70b92e-4b64-468e-867e-ebe7b1a418e0","Type":"ContainerDied","Data":"35a981aa92c3b76b4d4b68f2e62f967d6cf6c23f3912b858708ea235a123728d"}
Sep 04 22:19:33.839719 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.839697    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh" event={"ID":"ec8298c2-9f11-4d09-b997-917d6bf3941e","Type":"ContainerDied","Data":"cc68f26284e476ac375f0d08b42581790b6eb0aa243c5f5c7cbcd7086e871890"}
Sep 04 22:19:33.839848 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.839728    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"
Sep 04 22:19:33.846169 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.846154    2565 scope.go:117] "RemoveContainer" containerID="01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476"
Sep 04 22:19:33.846445 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:19:33.846427    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476\": container with ID starting with 01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476 not found: ID does not exist" containerID="01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476"
Sep 04 22:19:33.846514 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.846455    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476"} err="failed to get container status \"01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476\": rpc error: code = NotFound desc = could not find container \"01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476\": container with ID starting with 01a27e8b7bd31eb55d0687d574fe5111d75c7f8536d51cf1a9b0622cb9c72476 not found: ID does not exist"
Sep 04 22:19:33.846514 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.846472    2565 scope.go:117] "RemoveContainer" containerID="b9af8b13339ab56a4e4c4b8efd9ced2fec92df50aa218a23ba6c43a34dd6d8a1"
Sep 04 22:19:33.854106 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.854091    2565 scope.go:117] "RemoveContainer" containerID="644323e61174430e960f5c482aa552022b72548b1936f6ade51764f7b630470f"
Sep 04 22:19:33.858633 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.858612    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"]
Sep 04 22:19:33.865505 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.865481    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f2fbdf-68fdf785f8-7jjzv"]
Sep 04 22:19:33.874747 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.874724    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"]
Sep 04 22:19:33.877944 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.877922    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f2fbdf-996bc5647-qbkhh"]
Sep 04 22:19:33.887082 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.887060    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"]
Sep 04 22:19:33.890551 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:33.890530    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f2fbdf-58f9ff599b-rpvx6"]
Sep 04 22:19:35.389778 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:35.389747    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79a83595-5943-4f69-b2a9-3f7382fce96c" path="/var/lib/kubelet/pods/79a83595-5943-4f69-b2a9-3f7382fce96c/volumes"
Sep 04 22:19:35.390134 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:35.390100    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea70b92e-4b64-468e-867e-ebe7b1a418e0" path="/var/lib/kubelet/pods/ea70b92e-4b64-468e-867e-ebe7b1a418e0/volumes"
Sep 04 22:19:35.390426 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:35.390412    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec8298c2-9f11-4d09-b997-917d6bf3941e" path="/var/lib/kubelet/pods/ec8298c2-9f11-4d09-b997-917d6bf3941e/volumes"
Sep 04 22:19:50.068813 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.068774    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"]
Sep 04 22:19:50.069324 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.069054    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g" podUID="695961b3-e923-4370-b2ec-19f3f9621b7d" containerName="maas-api" containerID="cri-o://b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27" gracePeriod=30
Sep 04 22:19:50.130266 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.130233    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"]
Sep 04 22:19:50.130573 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.130543    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9" podUID="edd2a55b-d9f2-46c1-9877-70fca4d71b60" containerName="payload-processing" containerID="cri-o://6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1" gracePeriod=30
Sep 04 22:19:50.138572 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.138529    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"]
Sep 04 22:19:50.138781 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.138730    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd" podUID="05d94604-60e4-474c-9d27-5c9f2df1a216" containerName="payload-pre-processing" containerID="cri-o://a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2" gracePeriod=30
Sep 04 22:19:50.400740 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.400710    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:19:50.422904 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.422861    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:19:50.433556 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.433538    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:19:50.507868 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.507839    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd2a55b-d9f2-46c1-9877-70fca4d71b60-plugins-config-volume\") pod \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") "
Sep 04 22:19:50.508045 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.507878    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn8ln\" (UniqueName: \"kubernetes.io/projected/05d94604-60e4-474c-9d27-5c9f2df1a216-kube-api-access-xn8ln\") pod \"05d94604-60e4-474c-9d27-5c9f2df1a216\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") "
Sep 04 22:19:50.508117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/695961b3-e923-4370-b2ec-19f3f9621b7d-openshift-service-ca\") pod \"695961b3-e923-4370-b2ec-19f3f9621b7d\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") "
Sep 04 22:19:50.508117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508103    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05d94604-60e4-474c-9d27-5c9f2df1a216-plugins-config-volume\") pod \"05d94604-60e4-474c-9d27-5c9f2df1a216\" (UID: \"05d94604-60e4-474c-9d27-5c9f2df1a216\") "
Sep 04 22:19:50.508236 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508126    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg7v4\" (UniqueName: \"kubernetes.io/projected/695961b3-e923-4370-b2ec-19f3f9621b7d-kube-api-access-lg7v4\") pod \"695961b3-e923-4370-b2ec-19f3f9621b7d\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") "
Sep 04 22:19:50.508236 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508152    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-metrics-certs\") pod \"695961b3-e923-4370-b2ec-19f3f9621b7d\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") "
Sep 04 22:19:50.508236 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508187    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-maas-api-tls\") pod \"695961b3-e923-4370-b2ec-19f3f9621b7d\" (UID: \"695961b3-e923-4370-b2ec-19f3f9621b7d\") "
Sep 04 22:19:50.508236 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508201    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd2a55b-d9f2-46c1-9877-70fca4d71b60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "edd2a55b-d9f2-46c1-9877-70fca4d71b60" (UID: "edd2a55b-d9f2-46c1-9877-70fca4d71b60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:50.508236 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508235    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6td8\" (UniqueName: \"kubernetes.io/projected/edd2a55b-d9f2-46c1-9877-70fca4d71b60-kube-api-access-q6td8\") pod \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\" (UID: \"edd2a55b-d9f2-46c1-9877-70fca4d71b60\") "
Sep 04 22:19:50.508617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508441    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695961b3-e923-4370-b2ec-19f3f9621b7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "695961b3-e923-4370-b2ec-19f3f9621b7d" (UID: "695961b3-e923-4370-b2ec-19f3f9621b7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:50.508617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508454    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d94604-60e4-474c-9d27-5c9f2df1a216-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05d94604-60e4-474c-9d27-5c9f2df1a216" (UID: "05d94604-60e4-474c-9d27-5c9f2df1a216"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:50.508808 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508690    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/695961b3-e923-4370-b2ec-19f3f9621b7d-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.508808 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508711    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05d94604-60e4-474c-9d27-5c9f2df1a216-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.508808 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.508725    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd2a55b-d9f2-46c1-9877-70fca4d71b60-plugins-config-volume\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.510225 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.510194    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d94604-60e4-474c-9d27-5c9f2df1a216-kube-api-access-xn8ln" (OuterVolumeSpecName: "kube-api-access-xn8ln") pod "05d94604-60e4-474c-9d27-5c9f2df1a216" (UID: "05d94604-60e4-474c-9d27-5c9f2df1a216"). InnerVolumeSpecName "kube-api-access-xn8ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:50.510580 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.510555    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd2a55b-d9f2-46c1-9877-70fca4d71b60-kube-api-access-q6td8" (OuterVolumeSpecName: "kube-api-access-q6td8") pod "edd2a55b-d9f2-46c1-9877-70fca4d71b60" (UID: "edd2a55b-d9f2-46c1-9877-70fca4d71b60"). InnerVolumeSpecName "kube-api-access-q6td8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:50.510678 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.510560    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/695961b3-e923-4370-b2ec-19f3f9621b7d-kube-api-access-lg7v4" (OuterVolumeSpecName: "kube-api-access-lg7v4") pod "695961b3-e923-4370-b2ec-19f3f9621b7d" (UID: "695961b3-e923-4370-b2ec-19f3f9621b7d"). InnerVolumeSpecName "kube-api-access-lg7v4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:50.510907 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.510880    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "695961b3-e923-4370-b2ec-19f3f9621b7d" (UID: "695961b3-e923-4370-b2ec-19f3f9621b7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:19:50.511025 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.510962    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "695961b3-e923-4370-b2ec-19f3f9621b7d" (UID: "695961b3-e923-4370-b2ec-19f3f9621b7d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:19:50.609763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.609684    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn8ln\" (UniqueName: \"kubernetes.io/projected/05d94604-60e4-474c-9d27-5c9f2df1a216-kube-api-access-xn8ln\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.609763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.609711    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg7v4\" (UniqueName: \"kubernetes.io/projected/695961b3-e923-4370-b2ec-19f3f9621b7d-kube-api-access-lg7v4\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.609763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.609722    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.609763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.609730    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/695961b3-e923-4370-b2ec-19f3f9621b7d-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.609763 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.609738    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6td8\" (UniqueName: \"kubernetes.io/projected/edd2a55b-d9f2-46c1-9877-70fca4d71b60-kube-api-access-q6td8\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:19:50.921757 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.921678    2565 generic.go:356] "Generic (PLEG): container finished" podID="695961b3-e923-4370-b2ec-19f3f9621b7d" containerID="b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27" exitCode=0
Sep 04 22:19:50.921757 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.921724    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"
Sep 04 22:19:50.921757 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.921749    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g" event={"ID":"695961b3-e923-4370-b2ec-19f3f9621b7d","Type":"ContainerDied","Data":"b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27"}
Sep 04 22:19:50.921994 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.921797    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g" event={"ID":"695961b3-e923-4370-b2ec-19f3f9621b7d","Type":"ContainerDied","Data":"e12b87f92a8a61b411fa15aa2429d3dc0e7385b5113c56f84c89404a177d10c1"}
Sep 04 22:19:50.921994 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.921819    2565 scope.go:117] "RemoveContainer" containerID="b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27"
Sep 04 22:19:50.930536 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.930505    2565 generic.go:356] "Generic (PLEG): container finished" podID="edd2a55b-d9f2-46c1-9877-70fca4d71b60" containerID="6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1" exitCode=0
Sep 04 22:19:50.930690 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.930564    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"
Sep 04 22:19:50.930690 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.930612    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9" event={"ID":"edd2a55b-d9f2-46c1-9877-70fca4d71b60","Type":"ContainerDied","Data":"6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1"}
Sep 04 22:19:50.930810 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.930720    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9" event={"ID":"edd2a55b-d9f2-46c1-9877-70fca4d71b60","Type":"ContainerDied","Data":"83511280f531dbc116f2ddfb6cd8fd39f2b56f9bdd7d79cc3bfd0063aefb7302"}
Sep 04 22:19:50.932110 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.932083    2565 generic.go:356] "Generic (PLEG): container finished" podID="05d94604-60e4-474c-9d27-5c9f2df1a216" containerID="a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2" exitCode=0
Sep 04 22:19:50.932249 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.932130    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"
Sep 04 22:19:50.932249 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.932166    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd" event={"ID":"05d94604-60e4-474c-9d27-5c9f2df1a216","Type":"ContainerDied","Data":"a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2"}
Sep 04 22:19:50.932249 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.932204    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd" event={"ID":"05d94604-60e4-474c-9d27-5c9f2df1a216","Type":"ContainerDied","Data":"21bf580ebaf31065b9d18438163096d6d6cc98c816ec3e046889a11685f2c8c9"}
Sep 04 22:19:50.940352 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.940321    2565 scope.go:117] "RemoveContainer" containerID="b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27"
Sep 04 22:19:50.940630 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:19:50.940611    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27\": container with ID starting with b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27 not found: ID does not exist" containerID="b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27"
Sep 04 22:19:50.940715 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.940641    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27"} err="failed to get container status \"b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27\": rpc error: code = NotFound desc = could not find container \"b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27\": container with ID starting with b4d63c6ee1850281d7e717a177143877b11dc0de9b26602b3e8198aba8370b27 not found: ID does not exist"
Sep 04 22:19:50.940715 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.940662    2565 scope.go:117] "RemoveContainer" containerID="6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1"
Sep 04 22:19:50.948539 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.948521    2565 scope.go:117] "RemoveContainer" containerID="6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1"
Sep 04 22:19:50.948834 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:19:50.948811    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1\": container with ID starting with 6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1 not found: ID does not exist" containerID="6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1"
Sep 04 22:19:50.948926 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.948838    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1"} err="failed to get container status \"6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1\": rpc error: code = NotFound desc = could not find container \"6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1\": container with ID starting with 6fa9988202044d265e0c2b41357b5007e2058b8cda18f7e374ce353afe2006f1 not found: ID does not exist"
Sep 04 22:19:50.948926 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.948854    2565 scope.go:117] "RemoveContainer" containerID="a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2"
Sep 04 22:19:50.952642 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.952623    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"]
Sep 04 22:19:50.956943 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.956922    2565 scope.go:117] "RemoveContainer" containerID="a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2"
Sep 04 22:19:50.957209 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:19:50.957192    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2\": container with ID starting with a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2 not found: ID does not exist" containerID="a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2"
Sep 04 22:19:50.957276 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.957218    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2"} err="failed to get container status \"a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2\": rpc error: code = NotFound desc = could not find container \"a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2\": container with ID starting with a3d16ff0aac1240b9c60fc804d960aea66852a12069376b32bba5acc4370cbd2 not found: ID does not exist"
Sep 04 22:19:50.957804 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.957789    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6680c7-65b58489d8-5p67g"]
Sep 04 22:19:50.969437 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.969415    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"]
Sep 04 22:19:50.973182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.973161    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6680c7-79994f6b8c-d98g9"]
Sep 04 22:19:50.983551 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.983529    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"]
Sep 04 22:19:50.986767 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:50.986749    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6680c7-7c549bbfb6-9sltd"]
Sep 04 22:19:51.390429 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:51.390402    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05d94604-60e4-474c-9d27-5c9f2df1a216" path="/var/lib/kubelet/pods/05d94604-60e4-474c-9d27-5c9f2df1a216/volumes"
Sep 04 22:19:51.390789 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:51.390765    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="695961b3-e923-4370-b2ec-19f3f9621b7d" path="/var/lib/kubelet/pods/695961b3-e923-4370-b2ec-19f3f9621b7d/volumes"
Sep 04 22:19:51.391070 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:19:51.391057    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd2a55b-d9f2-46c1-9877-70fca4d71b60" path="/var/lib/kubelet/pods/edd2a55b-d9f2-46c1-9877-70fca4d71b60/volumes"
Sep 04 22:20:07.880758 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:07.880723    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"]
Sep 04 22:20:07.881409 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:07.880967    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" podUID="ca41b5aa-95f8-45d6-9d36-c889ff9e49af" containerName="maas-api" containerID="cri-o://b52fe622aa9e9bf171ca355109fdbfe1020a6c8c2f61a5e46ed70e4fc29d812f" gracePeriod=30
Sep 04 22:20:08.023620 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.023570    2565 generic.go:356] "Generic (PLEG): container finished" podID="ca41b5aa-95f8-45d6-9d36-c889ff9e49af" containerID="b52fe622aa9e9bf171ca355109fdbfe1020a6c8c2f61a5e46ed70e4fc29d812f" exitCode=0
Sep 04 22:20:08.023781 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.023641    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" event={"ID":"ca41b5aa-95f8-45d6-9d36-c889ff9e49af","Type":"ContainerDied","Data":"b52fe622aa9e9bf171ca355109fdbfe1020a6c8c2f61a5e46ed70e4fc29d812f"}
Sep 04 22:20:08.125115 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.124967    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:20:08.267908 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.267827    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-metrics-certs\") pod \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") "
Sep 04 22:20:08.267908 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.267884    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls\") pod \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") "
Sep 04 22:20:08.268152 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.267927    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmpc2\" (UniqueName: \"kubernetes.io/projected/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-kube-api-access-bmpc2\") pod \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") "
Sep 04 22:20:08.268152 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.267952    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-openshift-service-ca\") pod \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\" (UID: \"ca41b5aa-95f8-45d6-9d36-c889ff9e49af\") "
Sep 04 22:20:08.268444 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.268418    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca41b5aa-95f8-45d6-9d36-c889ff9e49af" (UID: "ca41b5aa-95f8-45d6-9d36-c889ff9e49af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:20:08.270071 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.270049    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ca41b5aa-95f8-45d6-9d36-c889ff9e49af" (UID: "ca41b5aa-95f8-45d6-9d36-c889ff9e49af"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:20:08.270174 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.270127    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca41b5aa-95f8-45d6-9d36-c889ff9e49af" (UID: "ca41b5aa-95f8-45d6-9d36-c889ff9e49af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:20:08.270228 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.270184    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-kube-api-access-bmpc2" (OuterVolumeSpecName: "kube-api-access-bmpc2") pod "ca41b5aa-95f8-45d6-9d36-c889ff9e49af" (UID: "ca41b5aa-95f8-45d6-9d36-c889ff9e49af"). InnerVolumeSpecName "kube-api-access-bmpc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:20:08.368910 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.368877    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.368910 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.368905    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.369101 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.368919    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmpc2\" (UniqueName: \"kubernetes.io/projected/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-kube-api-access-bmpc2\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.369101 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:08.368931    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca41b5aa-95f8-45d6-9d36-c889ff9e49af-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:09.030037 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:09.030000    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh" event={"ID":"ca41b5aa-95f8-45d6-9d36-c889ff9e49af","Type":"ContainerDied","Data":"252787ff7c8c093409edd34cdd48d849e05f6e3f95fadab2f9b14e17ab1d3360"}
Sep 04 22:20:09.030037 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:09.030019    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"
Sep 04 22:20:09.030567 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:09.030089    2565 scope.go:117] "RemoveContainer" containerID="b52fe622aa9e9bf171ca355109fdbfe1020a6c8c2f61a5e46ed70e4fc29d812f"
Sep 04 22:20:09.050794 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:09.050761    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"]
Sep 04 22:20:09.054474 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:09.054454    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-7a26c9-695d866777-nrwjh"]
Sep 04 22:20:09.389472 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:09.389444    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca41b5aa-95f8-45d6-9d36-c889ff9e49af" path="/var/lib/kubelet/pods/ca41b5aa-95f8-45d6-9d36-c889ff9e49af/volumes"
Sep 04 22:20:17.110153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.110108    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78b457d45d-jvwc8"]
Sep 04 22:20:17.110612 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.110306    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78b457d45d-jvwc8" podUID="ae07492c-0540-4230-a3e8-7b8cb681ca65" containerName="manager" containerID="cri-o://a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564" gracePeriod=30
Sep 04 22:20:17.351444 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.351423    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:20:17.454766 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.454694    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-metrics-certs\") pod \"ae07492c-0540-4230-a3e8-7b8cb681ca65\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") "
Sep 04 22:20:17.454766 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.454741    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-webhook-cert\") pod \"ae07492c-0540-4230-a3e8-7b8cb681ca65\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") "
Sep 04 22:20:17.454947 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.454798    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkqqm\" (UniqueName: \"kubernetes.io/projected/ae07492c-0540-4230-a3e8-7b8cb681ca65-kube-api-access-hkqqm\") pod \"ae07492c-0540-4230-a3e8-7b8cb681ca65\" (UID: \"ae07492c-0540-4230-a3e8-7b8cb681ca65\") "
Sep 04 22:20:17.456903 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.456870    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae07492c-0540-4230-a3e8-7b8cb681ca65" (UID: "ae07492c-0540-4230-a3e8-7b8cb681ca65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:20:17.457029 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.456910    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ae07492c-0540-4230-a3e8-7b8cb681ca65" (UID: "ae07492c-0540-4230-a3e8-7b8cb681ca65"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:20:17.457029 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.456963    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae07492c-0540-4230-a3e8-7b8cb681ca65-kube-api-access-hkqqm" (OuterVolumeSpecName: "kube-api-access-hkqqm") pod "ae07492c-0540-4230-a3e8-7b8cb681ca65" (UID: "ae07492c-0540-4230-a3e8-7b8cb681ca65"). InnerVolumeSpecName "kube-api-access-hkqqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:20:17.556359 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.556314    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkqqm\" (UniqueName: \"kubernetes.io/projected/ae07492c-0540-4230-a3e8-7b8cb681ca65-kube-api-access-hkqqm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:17.556359 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.556362    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:17.556559 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:17.556375    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae07492c-0540-4230-a3e8-7b8cb681ca65-webhook-cert\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:20:18.073102 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.073065    2565 generic.go:356] "Generic (PLEG): container finished" podID="ae07492c-0540-4230-a3e8-7b8cb681ca65" containerID="a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564" exitCode=0
Sep 04 22:20:18.073299 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.073109    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-jvwc8"
Sep 04 22:20:18.073299 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.073129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-jvwc8" event={"ID":"ae07492c-0540-4230-a3e8-7b8cb681ca65","Type":"ContainerDied","Data":"a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564"}
Sep 04 22:20:18.073299 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.073170    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-jvwc8" event={"ID":"ae07492c-0540-4230-a3e8-7b8cb681ca65","Type":"ContainerDied","Data":"61ea6a495491c605d62112bf9e7fa75a1e70eaa7af8eeb1b12ee432acfc1bea6"}
Sep 04 22:20:18.073299 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.073190    2565 scope.go:117] "RemoveContainer" containerID="a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564"
Sep 04 22:20:18.082394 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.082378    2565 scope.go:117] "RemoveContainer" containerID="a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564"
Sep 04 22:20:18.082651 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:20:18.082632    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564\": container with ID starting with a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564 not found: ID does not exist" containerID="a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564"
Sep 04 22:20:18.082709 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.082657    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564"} err="failed to get container status \"a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564\": rpc error: code = NotFound desc = could not find container \"a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564\": container with ID starting with a6bbcafafe7ac71c153be5581422f8bcd354595450f81b76b74a7d67884a2564 not found: ID does not exist"
Sep 04 22:20:18.094617 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.094594    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78b457d45d-jvwc8"]
Sep 04 22:20:18.098047 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:18.098028    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78b457d45d-jvwc8"]
Sep 04 22:20:19.390979 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:20:19.390944    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae07492c-0540-4230-a3e8-7b8cb681ca65" path="/var/lib/kubelet/pods/ae07492c-0540-4230-a3e8-7b8cb681ca65/volumes"
Sep 04 22:21:18.919172 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919136    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78b457d45d-nc9g7"]
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919529    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" containerName="payload-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919541    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" containerName="payload-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919551    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16c758da-3dc9-43b0-bd49-b4d40bae332a" containerName="payload-pre-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919557    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="16c758da-3dc9-43b0-bd49-b4d40bae332a" containerName="payload-pre-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919567    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edd2a55b-d9f2-46c1-9877-70fca4d71b60" containerName="payload-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919578    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd2a55b-d9f2-46c1-9877-70fca4d71b60" containerName="payload-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919587    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae07492c-0540-4230-a3e8-7b8cb681ca65" containerName="manager"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919592    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae07492c-0540-4230-a3e8-7b8cb681ca65" containerName="manager"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919606    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413a28b1-a348-40a2-a6dc-d5c2e27f6aac" containerName="maas-api"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919611    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="413a28b1-a348-40a2-a6dc-d5c2e27f6aac" containerName="maas-api"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919617    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d94604-60e4-474c-9d27-5c9f2df1a216" containerName="payload-pre-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919622    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d94604-60e4-474c-9d27-5c9f2df1a216" containerName="payload-pre-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919630    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="695961b3-e923-4370-b2ec-19f3f9621b7d" containerName="maas-api"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919635    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="695961b3-e923-4370-b2ec-19f3f9621b7d" containerName="maas-api"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919641    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164a6c3f-c745-4e39-a785-5f5f75f8844d" containerName="maas-api"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919650    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="164a6c3f-c745-4e39-a785-5f5f75f8844d" containerName="maas-api"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919660    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a83595-5943-4f69-b2a9-3f7382fce96c" containerName="payload-pre-processing"
Sep 04 22:21:18.919660 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919665    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a83595-5943-4f69-b2a9-3f7382fce96c" containerName="payload-pre-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919671    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca41b5aa-95f8-45d6-9d36-c889ff9e49af" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919676    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca41b5aa-95f8-45d6-9d36-c889ff9e49af" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919683    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea70b92e-4b64-468e-867e-ebe7b1a418e0" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919688    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea70b92e-4b64-468e-867e-ebe7b1a418e0" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919693    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec8298c2-9f11-4d09-b997-917d6bf3941e" containerName="payload-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919698    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec8298c2-9f11-4d09-b997-917d6bf3941e" containerName="payload-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919757    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="413a28b1-a348-40a2-a6dc-d5c2e27f6aac" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919764    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="164a6c3f-c745-4e39-a785-5f5f75f8844d" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919770    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="edd2a55b-d9f2-46c1-9877-70fca4d71b60" containerName="payload-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919774    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca41b5aa-95f8-45d6-9d36-c889ff9e49af" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919779    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec8298c2-9f11-4d09-b997-917d6bf3941e" containerName="payload-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919784    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f160840-b2bd-4e7d-b3e6-c1c22890f3ff" containerName="payload-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919789    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="79a83595-5943-4f69-b2a9-3f7382fce96c" containerName="payload-pre-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919793    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="16c758da-3dc9-43b0-bd49-b4d40bae332a" containerName="payload-pre-processing"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919798    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae07492c-0540-4230-a3e8-7b8cb681ca65" containerName="manager"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919814    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea70b92e-4b64-468e-867e-ebe7b1a418e0" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919824    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="695961b3-e923-4370-b2ec-19f3f9621b7d" containerName="maas-api"
Sep 04 22:21:18.920218 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.919831    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d94604-60e4-474c-9d27-5c9f2df1a216" containerName="payload-pre-processing"
Sep 04 22:21:18.922884 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.922868    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:18.926334 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.926302    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 04 22:21:18.926468 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.926360    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 04 22:21:18.926468 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.926372    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nwkwk\""
Sep 04 22:21:18.926468 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.926361    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 04 22:21:18.932670 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.932639    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78b457d45d-nc9g7"]
Sep 04 22:21:18.977935 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.977905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-metrics-certs\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:18.978087 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.977938    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rllxm\" (UniqueName: \"kubernetes.io/projected/ff37d590-df61-4262-b8e1-e19ae6658232-kube-api-access-rllxm\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:18.978087 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:18.977965    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-webhook-cert\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.078940 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.078907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-metrics-certs\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.079113 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.078945    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rllxm\" (UniqueName: \"kubernetes.io/projected/ff37d590-df61-4262-b8e1-e19ae6658232-kube-api-access-rllxm\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.079113 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.078976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-webhook-cert\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.081499 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.081466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-webhook-cert\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.081613 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.081567    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-metrics-certs\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.087128 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.087107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rllxm\" (UniqueName: \"kubernetes.io/projected/ff37d590-df61-4262-b8e1-e19ae6658232-kube-api-access-rllxm\") pod \"maas-controller-78b457d45d-nc9g7\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") " pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.234086 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.234006    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:19.502576 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:19.502552    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78b457d45d-nc9g7"]
Sep 04 22:21:19.505260 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:21:19.505230    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff37d590_df61_4262_b8e1_e19ae6658232.slice/crio-22874846e05f9b65a7037e6b92c0ce5e39d3b3e9fe4b7d01faff4b3e359f9cb0 WatchSource:0}: Error finding container 22874846e05f9b65a7037e6b92c0ce5e39d3b3e9fe4b7d01faff4b3e359f9cb0: Status 404 returned error can't find the container with id 22874846e05f9b65a7037e6b92c0ce5e39d3b3e9fe4b7d01faff4b3e359f9cb0
Sep 04 22:21:20.363523 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:20.363490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-nc9g7" event={"ID":"ff37d590-df61-4262-b8e1-e19ae6658232","Type":"ContainerStarted","Data":"b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3"}
Sep 04 22:21:20.363906 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:20.363568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-nc9g7" event={"ID":"ff37d590-df61-4262-b8e1-e19ae6658232","Type":"ContainerStarted","Data":"22874846e05f9b65a7037e6b92c0ce5e39d3b3e9fe4b7d01faff4b3e359f9cb0"}
Sep 04 22:21:20.363906 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:20.363611    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:20.381311 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:20.381269    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78b457d45d-nc9g7" podStartSLOduration=1.9430336270000002 podStartE2EDuration="2.381256081s" totalImagesPullingTime="438.222454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:21:20.380275075 +0000 UTC m=+1655.464659932" watchObservedRunningTime="2026-09-04 22:21:20.381256081 +0000 UTC m=+1655.465640924"
Sep 04 22:21:31.370003 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:31.369971    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:40.686182 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:40.686143    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78b457d45d-nc9g7"]
Sep 04 22:21:40.686595 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:40.686401    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78b457d45d-nc9g7" podUID="ff37d590-df61-4262-b8e1-e19ae6658232" containerName="manager" containerID="cri-o://b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3" gracePeriod=30
Sep 04 22:21:40.932659 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:40.932633    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:41.080029 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.079997    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rllxm\" (UniqueName: \"kubernetes.io/projected/ff37d590-df61-4262-b8e1-e19ae6658232-kube-api-access-rllxm\") pod \"ff37d590-df61-4262-b8e1-e19ae6658232\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") "
Sep 04 22:21:41.080188 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.080037    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-metrics-certs\") pod \"ff37d590-df61-4262-b8e1-e19ae6658232\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") "
Sep 04 22:21:41.080188 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.080095    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-webhook-cert\") pod \"ff37d590-df61-4262-b8e1-e19ae6658232\" (UID: \"ff37d590-df61-4262-b8e1-e19ae6658232\") "
Sep 04 22:21:41.082364 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.082311    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ff37d590-df61-4262-b8e1-e19ae6658232" (UID: "ff37d590-df61-4262-b8e1-e19ae6658232"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:21:41.082364 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.082325    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff37d590-df61-4262-b8e1-e19ae6658232-kube-api-access-rllxm" (OuterVolumeSpecName: "kube-api-access-rllxm") pod "ff37d590-df61-4262-b8e1-e19ae6658232" (UID: "ff37d590-df61-4262-b8e1-e19ae6658232"). InnerVolumeSpecName "kube-api-access-rllxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:21:41.082568 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.082415    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ff37d590-df61-4262-b8e1-e19ae6658232" (UID: "ff37d590-df61-4262-b8e1-e19ae6658232"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:21:41.181498 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.181462    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rllxm\" (UniqueName: \"kubernetes.io/projected/ff37d590-df61-4262-b8e1-e19ae6658232-kube-api-access-rllxm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:21:41.181498 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.181491    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:21:41.181498 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.181501    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff37d590-df61-4262-b8e1-e19ae6658232-webhook-cert\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:21:41.455666 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.455637    2565 generic.go:356] "Generic (PLEG): container finished" podID="ff37d590-df61-4262-b8e1-e19ae6658232" containerID="b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3" exitCode=0
Sep 04 22:21:41.455820 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.455683    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-nc9g7"
Sep 04 22:21:41.455820 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.455724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-nc9g7" event={"ID":"ff37d590-df61-4262-b8e1-e19ae6658232","Type":"ContainerDied","Data":"b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3"}
Sep 04 22:21:41.455820 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.455774    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-nc9g7" event={"ID":"ff37d590-df61-4262-b8e1-e19ae6658232","Type":"ContainerDied","Data":"22874846e05f9b65a7037e6b92c0ce5e39d3b3e9fe4b7d01faff4b3e359f9cb0"}
Sep 04 22:21:41.455820 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.455796    2565 scope.go:117] "RemoveContainer" containerID="b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3"
Sep 04 22:21:41.464750 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.464733    2565 scope.go:117] "RemoveContainer" containerID="b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3"
Sep 04 22:21:41.465038 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:21:41.465007    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3\": container with ID starting with b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3 not found: ID does not exist" containerID="b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3"
Sep 04 22:21:41.465038 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.465035    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3"} err="failed to get container status \"b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3\": rpc error: code = NotFound desc = could not find container \"b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3\": container with ID starting with b7a0a7cea90121e26c3e37ed01f7c4182c93df2234e975dcd2184d3c558d57b3 not found: ID does not exist"
Sep 04 22:21:41.472292 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.472270    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78b457d45d-nc9g7"]
Sep 04 22:21:41.476107 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:41.476087    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78b457d45d-nc9g7"]
Sep 04 22:21:42.257171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.257138    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78b457d45d-7b246"]
Sep 04 22:21:42.257562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.257514    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff37d590-df61-4262-b8e1-e19ae6658232" containerName="manager"
Sep 04 22:21:42.257562 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.257526    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff37d590-df61-4262-b8e1-e19ae6658232" containerName="manager"
Sep 04 22:21:42.257632 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.257595    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff37d590-df61-4262-b8e1-e19ae6658232" containerName="manager"
Sep 04 22:21:42.260642 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.260625    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.262979 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.262961    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 04 22:21:42.263918 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.263872    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 04 22:21:42.263918 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.263916    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nwkwk\""
Sep 04 22:21:42.264071 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.263897    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 04 22:21:42.268849 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.268828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78b457d45d-7b246"]
Sep 04 22:21:42.391438 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.391404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wpk8\" (UniqueName: \"kubernetes.io/projected/236c059c-3da3-4a9b-883f-f94ed0919cec-kube-api-access-9wpk8\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.391585 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.391448    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/236c059c-3da3-4a9b-883f-f94ed0919cec-metrics-certs\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.391585 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.391477    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/236c059c-3da3-4a9b-883f-f94ed0919cec-webhook-cert\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.492786 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.492753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wpk8\" (UniqueName: \"kubernetes.io/projected/236c059c-3da3-4a9b-883f-f94ed0919cec-kube-api-access-9wpk8\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.492978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.492801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/236c059c-3da3-4a9b-883f-f94ed0919cec-metrics-certs\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.492978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.492865    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/236c059c-3da3-4a9b-883f-f94ed0919cec-webhook-cert\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.495452 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.495429    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/236c059c-3da3-4a9b-883f-f94ed0919cec-webhook-cert\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.495452 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.495449    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/236c059c-3da3-4a9b-883f-f94ed0919cec-metrics-certs\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.500974 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.500957    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wpk8\" (UniqueName: \"kubernetes.io/projected/236c059c-3da3-4a9b-883f-f94ed0919cec-kube-api-access-9wpk8\") pod \"maas-controller-78b457d45d-7b246\" (UID: \"236c059c-3da3-4a9b-883f-f94ed0919cec\") " pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.572046 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.571949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:42.843768 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.843745    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78b457d45d-7b246"]
Sep 04 22:21:42.846054 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:21:42.846025    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236c059c_3da3_4a9b_883f_f94ed0919cec.slice/crio-09b2c134a4f3eab5b9b4f87f8e5039e83a3747a3ba45d52193cf02f33c44a069 WatchSource:0}: Error finding container 09b2c134a4f3eab5b9b4f87f8e5039e83a3747a3ba45d52193cf02f33c44a069: Status 404 returned error can't find the container with id 09b2c134a4f3eab5b9b4f87f8e5039e83a3747a3ba45d52193cf02f33c44a069
Sep 04 22:21:42.847880 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:42.847861    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:21:43.390980 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:43.390907    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff37d590-df61-4262-b8e1-e19ae6658232" path="/var/lib/kubelet/pods/ff37d590-df61-4262-b8e1-e19ae6658232/volumes"
Sep 04 22:21:43.468197 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:43.468165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-7b246" event={"ID":"236c059c-3da3-4a9b-883f-f94ed0919cec","Type":"ContainerStarted","Data":"63d697ae269a50033b227214c36a8f67bad03286269daddaa24d12a6cef4d68a"}
Sep 04 22:21:43.468401 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:43.468210    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78b457d45d-7b246" event={"ID":"236c059c-3da3-4a9b-883f-f94ed0919cec","Type":"ContainerStarted","Data":"09b2c134a4f3eab5b9b4f87f8e5039e83a3747a3ba45d52193cf02f33c44a069"}
Sep 04 22:21:43.468401 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:43.468241    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:21:43.485912 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:43.485865    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78b457d45d-7b246" podStartSLOduration=1.090326196 podStartE2EDuration="1.48585009s" totalImagesPullingTime="395.523894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:21:43.484947369 +0000 UTC m=+1678.569332214" watchObservedRunningTime="2026-09-04 22:21:43.48585009 +0000 UTC m=+1678.570234936"
Sep 04 22:21:45.616750 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:45.616720    2565 scope.go:117] "RemoveContainer" containerID="5df02776b2f1c7f287933690ac0a5e6efcac1372c536a5a27ff484cadc485ada"
Sep 04 22:21:54.476659 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:21:54.476629    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78b457d45d-7b246"
Sep 04 22:24:32.234124 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.234086    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"]
Sep 04 22:24:32.237586 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.237566    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.240019 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.239994    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ce8bb847-kserve-self-signed-certs\""
Sep 04 22:24:32.248634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.248608    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"]
Sep 04 22:24:32.380171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.380135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-dshm\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.380171 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.380172    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-tmp-dir\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.380435 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.380220    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-home\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.380435 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.380254    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32446df-ab9a-48a4-a9c9-07bfce137394-tls-certs\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.380435 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.380271    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-model-cache\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.380435 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.380376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxrt9\" (UniqueName: \"kubernetes.io/projected/f32446df-ab9a-48a4-a9c9-07bfce137394-kube-api-access-rxrt9\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.481853 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.481812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxrt9\" (UniqueName: \"kubernetes.io/projected/f32446df-ab9a-48a4-a9c9-07bfce137394-kube-api-access-rxrt9\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482039 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.481869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-dshm\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482039 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.481900    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-tmp-dir\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482039 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.481954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-home\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482039 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.481986    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32446df-ab9a-48a4-a9c9-07bfce137394-tls-certs\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482039 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.482015    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-model-cache\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482425 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.482399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-home\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482516 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.482445    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-tmp-dir\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.482516 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.482461    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-model-cache\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.484441 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.484388    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-dshm\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.484681 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.484660    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32446df-ab9a-48a4-a9c9-07bfce137394-tls-certs\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.488774 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.488751    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxrt9\" (UniqueName: \"kubernetes.io/projected/f32446df-ab9a-48a4-a9c9-07bfce137394-kube-api-access-rxrt9\") pod \"e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") " pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.548812 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.548783    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:32.820530 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:32.820450    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"]
Sep 04 22:24:32.823429 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:24:32.823402    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf32446df_ab9a_48a4_a9c9_07bfce137394.slice/crio-8846f43319f1fc6aad796a58e52dca104747548fc035bcaa763ce68f408e048a WatchSource:0}: Error finding container 8846f43319f1fc6aad796a58e52dca104747548fc035bcaa763ce68f408e048a: Status 404 returned error can't find the container with id 8846f43319f1fc6aad796a58e52dca104747548fc035bcaa763ce68f408e048a
Sep 04 22:24:33.223406 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:33.223367    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd" event={"ID":"f32446df-ab9a-48a4-a9c9-07bfce137394","Type":"ContainerStarted","Data":"c1487a3b1490a8ae7dc2fc1ccf98e372dbbe317000980f0d0b201685eba4beec"}
Sep 04 22:24:33.223575 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:33.223417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd" event={"ID":"f32446df-ab9a-48a4-a9c9-07bfce137394","Type":"ContainerStarted","Data":"8846f43319f1fc6aad796a58e52dca104747548fc035bcaa763ce68f408e048a"}
Sep 04 22:24:33.239940 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:33.239890    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd" podStartSLOduration=1.239876744 podStartE2EDuration="1.239876744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:24:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:24:33.238453551 +0000 UTC m=+1848.322838398" watchObservedRunningTime="2026-09-04 22:24:33.239876744 +0000 UTC m=+1848.324261587"
Sep 04 22:24:42.549514 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:42.549473    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:42.550000 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:42.549759    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:42.562444 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:42.562411    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:43.280588 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:43.280560    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:24:47.323504 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.323468    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"]
Sep 04 22:24:47.327153 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.327135    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.329310 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.329285    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ce8bb847-kserve-self-signed-certs\""
Sep 04 22:24:47.337554 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.337505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"]
Sep 04 22:24:47.416111 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.416071    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghxrv\" (UniqueName: \"kubernetes.io/projected/00b9b299-860b-47fb-86fa-043db82ffcf3-kube-api-access-ghxrv\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.416302 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.416168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-dshm\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.416302 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.416266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-tmp-dir\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.416458 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.416349    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-model-cache\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.416458 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.416389    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-home\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.416458 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.416427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00b9b299-860b-47fb-86fa-043db82ffcf3-tls-certs\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.517738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.517700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-tmp-dir\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.517738 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.517742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-model-cache\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.517971 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.517761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-home\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.517971 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.517783    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00b9b299-860b-47fb-86fa-043db82ffcf3-tls-certs\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.517971 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.517837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghxrv\" (UniqueName: \"kubernetes.io/projected/00b9b299-860b-47fb-86fa-043db82ffcf3-kube-api-access-ghxrv\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.517971 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.517869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-dshm\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.518193 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.518158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-tmp-dir\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.518259 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.518173    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-model-cache\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.518259 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.518236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-home\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.520403 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.520376    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-dshm\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.520664 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.520645    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00b9b299-860b-47fb-86fa-043db82ffcf3-tls-certs\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.525290 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.525260    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghxrv\" (UniqueName: \"kubernetes.io/projected/00b9b299-860b-47fb-86fa-043db82ffcf3-kube-api-access-ghxrv\") pod \"e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") " pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.638971 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.638935    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:47.910624 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:47.910601    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"]
Sep 04 22:24:47.912831 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:24:47.912806    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00b9b299_860b_47fb_86fa_043db82ffcf3.slice/crio-c5094434cc9b23062185fa58f86a3cf9b7943d5d6f1b61fc59dc3f5aedc4d28b WatchSource:0}: Error finding container c5094434cc9b23062185fa58f86a3cf9b7943d5d6f1b61fc59dc3f5aedc4d28b: Status 404 returned error can't find the container with id c5094434cc9b23062185fa58f86a3cf9b7943d5d6f1b61fc59dc3f5aedc4d28b
Sep 04 22:24:48.295283 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:48.295182    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m" event={"ID":"00b9b299-860b-47fb-86fa-043db82ffcf3","Type":"ContainerStarted","Data":"e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9"}
Sep 04 22:24:48.295283 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:48.295223    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m" event={"ID":"00b9b299-860b-47fb-86fa-043db82ffcf3","Type":"ContainerStarted","Data":"c5094434cc9b23062185fa58f86a3cf9b7943d5d6f1b61fc59dc3f5aedc4d28b"}
Sep 04 22:24:48.326956 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:48.326906    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m" podStartSLOduration=1.326887608 podStartE2EDuration="1.326887608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:24:48.325734241 +0000 UTC m=+1863.410119108" watchObservedRunningTime="2026-09-04 22:24:48.326887608 +0000 UTC m=+1863.411272453"
Sep 04 22:24:57.640131 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:57.640093    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:57.640640 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:57.640147    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:57.652560 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:57.652534    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:24:58.352895 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:24:58.352860    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:25:20.307274 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:20.307238    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"]
Sep 04 22:25:21.779883 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:21.779844    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"]
Sep 04 22:25:35.328429 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.328362    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd" podUID="f32446df-ab9a-48a4-a9c9-07bfce137394" containerName="main" containerID="cri-o://c1487a3b1490a8ae7dc2fc1ccf98e372dbbe317000980f0d0b201685eba4beec" gracePeriod=45
Sep 04 22:25:35.515119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.515087    2565 generic.go:356] "Generic (PLEG): container finished" podID="f32446df-ab9a-48a4-a9c9-07bfce137394" containerID="c1487a3b1490a8ae7dc2fc1ccf98e372dbbe317000980f0d0b201685eba4beec" exitCode=0
Sep 04 22:25:35.515264 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.515162    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd" event={"ID":"f32446df-ab9a-48a4-a9c9-07bfce137394","Type":"ContainerDied","Data":"c1487a3b1490a8ae7dc2fc1ccf98e372dbbe317000980f0d0b201685eba4beec"}
Sep 04 22:25:35.569552 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.569529    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:25:35.654829 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.654740    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-dshm\") pod \"f32446df-ab9a-48a4-a9c9-07bfce137394\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") "
Sep 04 22:25:35.654829 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.654791    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxrt9\" (UniqueName: \"kubernetes.io/projected/f32446df-ab9a-48a4-a9c9-07bfce137394-kube-api-access-rxrt9\") pod \"f32446df-ab9a-48a4-a9c9-07bfce137394\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") "
Sep 04 22:25:35.654829 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.654817    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-home\") pod \"f32446df-ab9a-48a4-a9c9-07bfce137394\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") "
Sep 04 22:25:35.655123 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.654857    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-tmp-dir\") pod \"f32446df-ab9a-48a4-a9c9-07bfce137394\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") "
Sep 04 22:25:35.655123 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655058    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32446df-ab9a-48a4-a9c9-07bfce137394-tls-certs\") pod \"f32446df-ab9a-48a4-a9c9-07bfce137394\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") "
Sep 04 22:25:35.655123 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655069    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-home" (OuterVolumeSpecName: "home") pod "f32446df-ab9a-48a4-a9c9-07bfce137394" (UID: "f32446df-ab9a-48a4-a9c9-07bfce137394"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:35.655123 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655111    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f32446df-ab9a-48a4-a9c9-07bfce137394" (UID: "f32446df-ab9a-48a4-a9c9-07bfce137394"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:35.655320 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655154    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-model-cache\") pod \"f32446df-ab9a-48a4-a9c9-07bfce137394\" (UID: \"f32446df-ab9a-48a4-a9c9-07bfce137394\") "
Sep 04 22:25:35.655407 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655393    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-model-cache" (OuterVolumeSpecName: "model-cache") pod "f32446df-ab9a-48a4-a9c9-07bfce137394" (UID: "f32446df-ab9a-48a4-a9c9-07bfce137394"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:35.655466 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655436    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:35.655466 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.655449    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:35.657045 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.657015    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-dshm" (OuterVolumeSpecName: "dshm") pod "f32446df-ab9a-48a4-a9c9-07bfce137394" (UID: "f32446df-ab9a-48a4-a9c9-07bfce137394"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:35.657193 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.657174    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32446df-ab9a-48a4-a9c9-07bfce137394-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f32446df-ab9a-48a4-a9c9-07bfce137394" (UID: "f32446df-ab9a-48a4-a9c9-07bfce137394"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:25:35.657238 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.657204    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32446df-ab9a-48a4-a9c9-07bfce137394-kube-api-access-rxrt9" (OuterVolumeSpecName: "kube-api-access-rxrt9") pod "f32446df-ab9a-48a4-a9c9-07bfce137394" (UID: "f32446df-ab9a-48a4-a9c9-07bfce137394"). InnerVolumeSpecName "kube-api-access-rxrt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:25:35.755978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.755946    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxrt9\" (UniqueName: \"kubernetes.io/projected/f32446df-ab9a-48a4-a9c9-07bfce137394-kube-api-access-rxrt9\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:35.755978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.755973    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32446df-ab9a-48a4-a9c9-07bfce137394-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:35.755978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.755983    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:35.756181 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:35.755991    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32446df-ab9a-48a4-a9c9-07bfce137394-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:36.522008 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:36.521978    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"
Sep 04 22:25:36.522495 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:36.521978    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd" event={"ID":"f32446df-ab9a-48a4-a9c9-07bfce137394","Type":"ContainerDied","Data":"8846f43319f1fc6aad796a58e52dca104747548fc035bcaa763ce68f408e048a"}
Sep 04 22:25:36.522495 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:36.522128    2565 scope.go:117] "RemoveContainer" containerID="c1487a3b1490a8ae7dc2fc1ccf98e372dbbe317000980f0d0b201685eba4beec"
Sep 04 22:25:36.541759 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:36.541737    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"]
Sep 04 22:25:36.545578 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:36.545547    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ce8bb847-kserve-776d9b894c-klzfd"]
Sep 04 22:25:36.799608 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:36.799527    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m" podUID="00b9b299-860b-47fb-86fa-043db82ffcf3" containerName="main" containerID="cri-o://e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9" gracePeriod=45
Sep 04 22:25:37.049079 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.049056    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:25:37.169011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.168974    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghxrv\" (UniqueName: \"kubernetes.io/projected/00b9b299-860b-47fb-86fa-043db82ffcf3-kube-api-access-ghxrv\") pod \"00b9b299-860b-47fb-86fa-043db82ffcf3\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") "
Sep 04 22:25:37.169011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169028    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-model-cache\") pod \"00b9b299-860b-47fb-86fa-043db82ffcf3\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") "
Sep 04 22:25:37.169266 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169067    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-dshm\") pod \"00b9b299-860b-47fb-86fa-043db82ffcf3\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") "
Sep 04 22:25:37.169266 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169172    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-home\") pod \"00b9b299-860b-47fb-86fa-043db82ffcf3\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") "
Sep 04 22:25:37.169266 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169253    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-tmp-dir\") pod \"00b9b299-860b-47fb-86fa-043db82ffcf3\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") "
Sep 04 22:25:37.169465 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169292    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-model-cache" (OuterVolumeSpecName: "model-cache") pod "00b9b299-860b-47fb-86fa-043db82ffcf3" (UID: "00b9b299-860b-47fb-86fa-043db82ffcf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:37.169465 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169332    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00b9b299-860b-47fb-86fa-043db82ffcf3-tls-certs\") pod \"00b9b299-860b-47fb-86fa-043db82ffcf3\" (UID: \"00b9b299-860b-47fb-86fa-043db82ffcf3\") "
Sep 04 22:25:37.169465 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169441    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-home" (OuterVolumeSpecName: "home") pod "00b9b299-860b-47fb-86fa-043db82ffcf3" (UID: "00b9b299-860b-47fb-86fa-043db82ffcf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:37.169611 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169568    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00b9b299-860b-47fb-86fa-043db82ffcf3" (UID: "00b9b299-860b-47fb-86fa-043db82ffcf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:37.169810 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169720    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-model-cache\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:37.170045 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169876    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-home\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:37.170045 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.169890    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-tmp-dir\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:37.171263 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.171235    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-dshm" (OuterVolumeSpecName: "dshm") pod "00b9b299-860b-47fb-86fa-043db82ffcf3" (UID: "00b9b299-860b-47fb-86fa-043db82ffcf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:25:37.171399 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.171322    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b9b299-860b-47fb-86fa-043db82ffcf3-kube-api-access-ghxrv" (OuterVolumeSpecName: "kube-api-access-ghxrv") pod "00b9b299-860b-47fb-86fa-043db82ffcf3" (UID: "00b9b299-860b-47fb-86fa-043db82ffcf3"). InnerVolumeSpecName "kube-api-access-ghxrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:25:37.171455 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.171431    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b9b299-860b-47fb-86fa-043db82ffcf3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00b9b299-860b-47fb-86fa-043db82ffcf3" (UID: "00b9b299-860b-47fb-86fa-043db82ffcf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:25:37.270940 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.270905    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00b9b299-860b-47fb-86fa-043db82ffcf3-tls-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:37.270940 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.270938    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghxrv\" (UniqueName: \"kubernetes.io/projected/00b9b299-860b-47fb-86fa-043db82ffcf3-kube-api-access-ghxrv\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:37.271127 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.270953    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00b9b299-860b-47fb-86fa-043db82ffcf3-dshm\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:37.389950 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.389908    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f32446df-ab9a-48a4-a9c9-07bfce137394" path="/var/lib/kubelet/pods/f32446df-ab9a-48a4-a9c9-07bfce137394/volumes"
Sep 04 22:25:37.527026 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.526934    2565 generic.go:356] "Generic (PLEG): container finished" podID="00b9b299-860b-47fb-86fa-043db82ffcf3" containerID="e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9" exitCode=0
Sep 04 22:25:37.527026 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.526991    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"
Sep 04 22:25:37.527544 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.527031    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m" event={"ID":"00b9b299-860b-47fb-86fa-043db82ffcf3","Type":"ContainerDied","Data":"e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9"}
Sep 04 22:25:37.527544 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.527078    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m" event={"ID":"00b9b299-860b-47fb-86fa-043db82ffcf3","Type":"ContainerDied","Data":"c5094434cc9b23062185fa58f86a3cf9b7943d5d6f1b61fc59dc3f5aedc4d28b"}
Sep 04 22:25:37.527544 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.527095    2565 scope.go:117] "RemoveContainer" containerID="e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9"
Sep 04 22:25:37.535294 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.535278    2565 scope.go:117] "RemoveContainer" containerID="e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9"
Sep 04 22:25:37.535544 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:25:37.535525    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9\": container with ID starting with e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9 not found: ID does not exist" containerID="e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9"
Sep 04 22:25:37.535606 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.535551    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9"} err="failed to get container status \"e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9\": rpc error: code = NotFound desc = could not find container \"e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9\": container with ID starting with e2cbe6547743024d8f9ae669d95c71e4e48245da8f4b654bbf5283f80bc985d9 not found: ID does not exist"
Sep 04 22:25:37.541756 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.541734    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"]
Sep 04 22:25:37.545132 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:37.545110    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ce8bb847-kserve-dd57d698c-pvg9m"]
Sep 04 22:25:39.390530 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.390499    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b9b299-860b-47fb-86fa-043db82ffcf3" path="/var/lib/kubelet/pods/00b9b299-860b-47fb-86fa-043db82ffcf3/volumes"
Sep 04 22:25:39.965204 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965172    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"]
Sep 04 22:25:39.965548 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965534    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32446df-ab9a-48a4-a9c9-07bfce137394" containerName="main"
Sep 04 22:25:39.965604 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965550    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32446df-ab9a-48a4-a9c9-07bfce137394" containerName="main"
Sep 04 22:25:39.965604 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965570    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b9b299-860b-47fb-86fa-043db82ffcf3" containerName="main"
Sep 04 22:25:39.965604 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965576    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b9b299-860b-47fb-86fa-043db82ffcf3" containerName="main"
Sep 04 22:25:39.965704 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965627    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b9b299-860b-47fb-86fa-043db82ffcf3" containerName="main"
Sep 04 22:25:39.965704 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.965638    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f32446df-ab9a-48a4-a9c9-07bfce137394" containerName="main"
Sep 04 22:25:39.969972 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.969954    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:39.978404 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:39.978197    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"]
Sep 04 22:25:40.096872 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.096838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-552xv\" (UniqueName: \"kubernetes.io/projected/075b8266-1f69-4f43-9222-5ee210e9d78f-kube-api-access-552xv\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.097058 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.096886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075b8266-1f69-4f43-9222-5ee210e9d78f-openshift-service-ca\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.097058 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.096920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/075b8266-1f69-4f43-9222-5ee210e9d78f-metrics-certs\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.097058 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.096938    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075b8266-1f69-4f43-9222-5ee210e9d78f-maas-api-tls\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.198119 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.198079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-552xv\" (UniqueName: \"kubernetes.io/projected/075b8266-1f69-4f43-9222-5ee210e9d78f-kube-api-access-552xv\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.198279 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.198140    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075b8266-1f69-4f43-9222-5ee210e9d78f-openshift-service-ca\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.198279 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.198192    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/075b8266-1f69-4f43-9222-5ee210e9d78f-metrics-certs\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.198279 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.198222    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075b8266-1f69-4f43-9222-5ee210e9d78f-maas-api-tls\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.198960 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.198934    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075b8266-1f69-4f43-9222-5ee210e9d78f-openshift-service-ca\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.200805 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.200780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/075b8266-1f69-4f43-9222-5ee210e9d78f-metrics-certs\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.200908 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.200848    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075b8266-1f69-4f43-9222-5ee210e9d78f-maas-api-tls\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.209982 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.209962    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-552xv\" (UniqueName: \"kubernetes.io/projected/075b8266-1f69-4f43-9222-5ee210e9d78f-kube-api-access-552xv\") pod \"maas-api-755dcb75c7-qrtdh\" (UID: \"075b8266-1f69-4f43-9222-5ee210e9d78f\") " pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.282439 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.282362    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:40.559947 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:40.559922    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"]
Sep 04 22:25:40.561772 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:25:40.561745    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075b8266_1f69_4f43_9222_5ee210e9d78f.slice/crio-0621141a830d00d03449eff72911a37f7b8ecdd5c57976412d9e615d7b787971 WatchSource:0}: Error finding container 0621141a830d00d03449eff72911a37f7b8ecdd5c57976412d9e615d7b787971: Status 404 returned error can't find the container with id 0621141a830d00d03449eff72911a37f7b8ecdd5c57976412d9e615d7b787971
Sep 04 22:25:41.548213 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:41.548172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh" event={"ID":"075b8266-1f69-4f43-9222-5ee210e9d78f","Type":"ContainerStarted","Data":"5cfa960a520cb592cc9927d8222764d477ac1cd20e722bb737d9b97eede097dc"}
Sep 04 22:25:41.548417 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:41.548233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh" event={"ID":"075b8266-1f69-4f43-9222-5ee210e9d78f","Type":"ContainerStarted","Data":"0621141a830d00d03449eff72911a37f7b8ecdd5c57976412d9e615d7b787971"}
Sep 04 22:25:41.548417 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:41.548259    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:41.565120 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:41.565070    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh" podStartSLOduration=2.140167732 podStartE2EDuration="2.565055955s" totalImagesPullingTime="424.888223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:25:41.56370485 +0000 UTC m=+1916.648089696" watchObservedRunningTime="2026-09-04 22:25:41.565055955 +0000 UTC m=+1916.649440799"
Sep 04 22:25:45.933842 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:45.933761    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-786d4f99d5-kddnf_f9430c8a-e334-4f73-b71f-2ff7d5964b6b/authorino/0.log"
Sep 04 22:25:47.558511 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:47.558476    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-755dcb75c7-qrtdh"
Sep 04 22:25:47.666273 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:47.666244    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"]
Sep 04 22:25:47.666454 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:47.666439    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" podUID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" containerName="maas-api" containerID="cri-o://2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c" gracePeriod=30
Sep 04 22:25:48.740979 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.740942    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" podUID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.28:8443/health\": dial tcp 10.133.0.28:8443: connect: connection refused"
Sep 04 22:25:48.908534 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.908508    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:25:48.972978 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.972908    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8pf9\" (UniqueName: \"kubernetes.io/projected/c960dee6-9338-4b91-b9d1-1351e3f0bb07-kube-api-access-p8pf9\") pod \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") "
Sep 04 22:25:48.973117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.972978    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") pod \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") "
Sep 04 22:25:48.973117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.973076    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-maas-api-tls\") pod \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") "
Sep 04 22:25:48.973117 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.973108    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c960dee6-9338-4b91-b9d1-1351e3f0bb07-openshift-service-ca\") pod \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\" (UID: \"c960dee6-9338-4b91-b9d1-1351e3f0bb07\") "
Sep 04 22:25:48.973503 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.973476    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c960dee6-9338-4b91-b9d1-1351e3f0bb07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c960dee6-9338-4b91-b9d1-1351e3f0bb07" (UID: "c960dee6-9338-4b91-b9d1-1351e3f0bb07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:25:48.975177 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.975148    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c960dee6-9338-4b91-b9d1-1351e3f0bb07" (UID: "c960dee6-9338-4b91-b9d1-1351e3f0bb07"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:25:48.975177 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.975147    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c960dee6-9338-4b91-b9d1-1351e3f0bb07-kube-api-access-p8pf9" (OuterVolumeSpecName: "kube-api-access-p8pf9") pod "c960dee6-9338-4b91-b9d1-1351e3f0bb07" (UID: "c960dee6-9338-4b91-b9d1-1351e3f0bb07"). InnerVolumeSpecName "kube-api-access-p8pf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:25:48.975309 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:48.975187    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c960dee6-9338-4b91-b9d1-1351e3f0bb07" (UID: "c960dee6-9338-4b91-b9d1-1351e3f0bb07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:25:49.074264 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.074236    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-maas-api-tls\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:49.074264 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.074264    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c960dee6-9338-4b91-b9d1-1351e3f0bb07-openshift-service-ca\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:49.074463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.074274    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8pf9\" (UniqueName: \"kubernetes.io/projected/c960dee6-9338-4b91-b9d1-1351e3f0bb07-kube-api-access-p8pf9\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:49.074463 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.074283    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c960dee6-9338-4b91-b9d1-1351e3f0bb07-metrics-certs\") on node \"ip-10-0-133-119.ec2.internal\" DevicePath \"\""
Sep 04 22:25:49.586477 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.586444    2565 generic.go:356] "Generic (PLEG): container finished" podID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" containerID="2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c" exitCode=0
Sep 04 22:25:49.586634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.586490    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"
Sep 04 22:25:49.586634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.586511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" event={"ID":"c960dee6-9338-4b91-b9d1-1351e3f0bb07","Type":"ContainerDied","Data":"2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c"}
Sep 04 22:25:49.586634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.586572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt" event={"ID":"c960dee6-9338-4b91-b9d1-1351e3f0bb07","Type":"ContainerDied","Data":"9079286e6fb39478cd813e8074757326848964555a4fa9fd062452e27dc1960c"}
Sep 04 22:25:49.586634 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.586602    2565 scope.go:117] "RemoveContainer" containerID="2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c"
Sep 04 22:25:49.601120 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.601099    2565 scope.go:117] "RemoveContainer" containerID="2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c"
Sep 04 22:25:49.601435 ip-10-0-133-119 kubenswrapper[2565]: E0904 22:25:49.601413    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c\": container with ID starting with 2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c not found: ID does not exist" containerID="2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c"
Sep 04 22:25:49.601501 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.601444    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c"} err="failed to get container status \"2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c\": rpc error: code = NotFound desc = could not find container \"2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c\": container with ID starting with 2cc594d77d51764df20e593bfded680595e3226cba3963ccc3b2de205ebe604c not found: ID does not exist"
Sep 04 22:25:49.606729 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.606707    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"]
Sep 04 22:25:49.609945 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:49.609925    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-65db659b7c-mxnkt"]
Sep 04 22:25:51.323350 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:51.323309    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vvz9x_06fa6853-e605-4a9a-94ca-a3f8d5218f95/manager/0.log"
Sep 04 22:25:51.390734 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:51.390700    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" path="/var/lib/kubelet/pods/c960dee6-9338-4b91-b9d1-1351e3f0bb07/volumes"
Sep 04 22:25:51.449395 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:51.449370    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-78b457d45d-7b246_236c059c-3da3-4a9b-883f-f94ed0919cec/manager/0.log"
Sep 04 22:25:51.786011 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:51.785980    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-5hmtg_704c64d7-cdd6-4574-9f9f-90bcb7be3144/copy-manifests/0.log"
Sep 04 22:25:51.792064 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:51.792041    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-5hmtg_704c64d7-cdd6-4574-9f9f-90bcb7be3144/manager/0.log"
Sep 04 22:25:52.635667 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:52.635638    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-755dcb75c7-qrtdh_075b8266-1f69-4f43-9222-5ee210e9d78f/maas-api/0.log"
Sep 04 22:25:52.858862 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:52.858834    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-qv7n9_8131eac8-04a1-466b-a932-320b8c9e6c4e/postgres/0.log"
Sep 04 22:25:53.638847 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:53.638817    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-786d4f99d5-kddnf_f9430c8a-e334-4f73-b71f-2ff7d5964b6b/authorino/0.log"
Sep 04 22:25:55.053048 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:55.053019    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85db8bc976-sc7bd_c7ce2525-b7c3-41a0-82ee-3226a31ff455/kube-auth-proxy/0.log"
Sep 04 22:25:55.305506 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:55.305430    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xqq_441f14f6-59e1-4c21-8441-a8bcffcb193d/payload-pre-processing/0.log"
Sep 04 22:25:55.902451 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:25:55.902422    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5td5p_9f0826af-7ddb-41c7-a7a3-bf3cd0271119/main/0.log"
Sep 04 22:26:01.695805 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.695765    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gqs8f/must-gather-cvkkq"]
Sep 04 22:26:01.696199 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.696133    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" containerName="maas-api"
Sep 04 22:26:01.696199 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.696145    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" containerName="maas-api"
Sep 04 22:26:01.696284 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.696215    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c960dee6-9338-4b91-b9d1-1351e3f0bb07" containerName="maas-api"
Sep 04 22:26:01.700707 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.700686    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:01.702914 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.702891    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqs8f\"/\"kube-root-ca.crt\""
Sep 04 22:26:01.702914 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.702907    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gqs8f\"/\"default-dockercfg-725pc\""
Sep 04 22:26:01.703081 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.702947    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqs8f\"/\"openshift-service-ca.crt\""
Sep 04 22:26:01.706934 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.706912    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqs8f/must-gather-cvkkq"]
Sep 04 22:26:01.884216 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.884178    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rs4x\" (UniqueName: \"kubernetes.io/projected/2aaddfe8-bb8a-4565-97fa-154d43c8573e-kube-api-access-2rs4x\") pod \"must-gather-cvkkq\" (UID: \"2aaddfe8-bb8a-4565-97fa-154d43c8573e\") " pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:01.884422 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.884236    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2aaddfe8-bb8a-4565-97fa-154d43c8573e-must-gather-output\") pod \"must-gather-cvkkq\" (UID: \"2aaddfe8-bb8a-4565-97fa-154d43c8573e\") " pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:01.985114 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.985011    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rs4x\" (UniqueName: \"kubernetes.io/projected/2aaddfe8-bb8a-4565-97fa-154d43c8573e-kube-api-access-2rs4x\") pod \"must-gather-cvkkq\" (UID: \"2aaddfe8-bb8a-4565-97fa-154d43c8573e\") " pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:01.985114 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.985087    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2aaddfe8-bb8a-4565-97fa-154d43c8573e-must-gather-output\") pod \"must-gather-cvkkq\" (UID: \"2aaddfe8-bb8a-4565-97fa-154d43c8573e\") " pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:01.985485 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.985463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2aaddfe8-bb8a-4565-97fa-154d43c8573e-must-gather-output\") pod \"must-gather-cvkkq\" (UID: \"2aaddfe8-bb8a-4565-97fa-154d43c8573e\") " pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:01.992661 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:01.992635    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rs4x\" (UniqueName: \"kubernetes.io/projected/2aaddfe8-bb8a-4565-97fa-154d43c8573e-kube-api-access-2rs4x\") pod \"must-gather-cvkkq\" (UID: \"2aaddfe8-bb8a-4565-97fa-154d43c8573e\") " pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:02.011653 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:02.011628    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqs8f/must-gather-cvkkq"
Sep 04 22:26:02.294275 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:02.294249    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqs8f/must-gather-cvkkq"]
Sep 04 22:26:02.295936 ip-10-0-133-119 kubenswrapper[2565]: W0904 22:26:02.295910    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2aaddfe8_bb8a_4565_97fa_154d43c8573e.slice/crio-c3e6858c1b20abb6194f48e5cec6c54d8035b18b24976b93343d4a19a687d8d1 WatchSource:0}: Error finding container c3e6858c1b20abb6194f48e5cec6c54d8035b18b24976b93343d4a19a687d8d1: Status 404 returned error can't find the container with id c3e6858c1b20abb6194f48e5cec6c54d8035b18b24976b93343d4a19a687d8d1
Sep 04 22:26:02.648418 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:02.648368    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/must-gather-cvkkq" event={"ID":"2aaddfe8-bb8a-4565-97fa-154d43c8573e","Type":"ContainerStarted","Data":"c3e6858c1b20abb6194f48e5cec6c54d8035b18b24976b93343d4a19a687d8d1"}
Sep 04 22:26:03.655776 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:03.655738    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/must-gather-cvkkq" event={"ID":"2aaddfe8-bb8a-4565-97fa-154d43c8573e","Type":"ContainerStarted","Data":"04362f355f63bda85375aa52859dc6528eb1a02e14b1b5a10c1871a7be099454"}
Sep 04 22:26:03.656246 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:03.655791    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/must-gather-cvkkq" event={"ID":"2aaddfe8-bb8a-4565-97fa-154d43c8573e","Type":"ContainerStarted","Data":"73f6510a3695f2f6f86cfab51fcd1a230fe24910f26f9bf66e5a12af6f8b2575"}
Sep 04 22:26:03.673087 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:03.673017    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gqs8f/must-gather-cvkkq" podStartSLOduration=1.810236038 podStartE2EDuration="2.672995865s" totalImagesPullingTime="862.759827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:26:03.670768389 +0000 UTC m=+1938.755153234" watchObservedRunningTime="2026-09-04 22:26:03.672995865 +0000 UTC m=+1938.757380710"
Sep 04 22:26:04.751937 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:04.751898    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f96gh_c6b2f1f2-f011-4d5e-b80c-f15c6add8565/global-pull-secret-syncer/0.log"
Sep 04 22:26:04.875650 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:04.875619    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pmzjn_0b8f1fce-1378-4ae4-9b93-bbde1a048fa6/konnectivity-agent/0.log"
Sep 04 22:26:04.944056 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:04.944024    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-119.ec2.internal_85ce9a2a03055b1e4010209d2f26585d/haproxy/0.log"
Sep 04 22:26:09.644363 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:09.644306    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-786d4f99d5-kddnf_f9430c8a-e334-4f73-b71f-2ff7d5964b6b/authorino/0.log"
Sep 04 22:26:11.716423 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:11.716381    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85d8b975b8-vxjbq_cd2dab44-2ef9-4387-aea4-9460ad57b7c3/metrics-server/0.log"
Sep 04 22:26:11.743462 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:11.743376    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-r6x9k_4c993cc6-0448-44ab-9dbb-aee44f55e7fd/monitoring-plugin/0.log"
Sep 04 22:26:11.772751 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:11.772724    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k6qht_29723d57-fa71-4252-8517-89db77537ec3/node-exporter/0.log"
Sep 04 22:26:11.798288 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:11.798258    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k6qht_29723d57-fa71-4252-8517-89db77537ec3/kube-rbac-proxy/0.log"
Sep 04 22:26:11.818894 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:11.818868    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k6qht_29723d57-fa71-4252-8517-89db77537ec3/init-textfile/0.log"
Sep 04 22:26:12.092409 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.092378    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/prometheus/0.log"
Sep 04 22:26:12.110885 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.110775    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/config-reloader/0.log"
Sep 04 22:26:12.132176 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.132147    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/thanos-sidecar/0.log"
Sep 04 22:26:12.153569 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.153514    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/kube-rbac-proxy-web/0.log"
Sep 04 22:26:12.173441 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.173414    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/kube-rbac-proxy/0.log"
Sep 04 22:26:12.192455 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.192428    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/kube-rbac-proxy-thanos/0.log"
Sep 04 22:26:12.211747 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.211717    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9ca58a9a-53bc-4ba8-84d8-23e112e1f86f/init-config-reloader/0.log"
Sep 04 22:26:12.239823 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.239781    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-5qkbg_85d075e4-6427-45bf-9553-65175191f48b/prometheus-operator/0.log"
Sep 04 22:26:12.259103 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.259026    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-5qkbg_85d075e4-6427-45bf-9553-65175191f48b/kube-rbac-proxy/0.log"
Sep 04 22:26:12.286329 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.286302    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-457w5_95f7c41a-75b6-403a-91cc-73ce1dafc5e6/prometheus-operator-admission-webhook/0.log"
Sep 04 22:26:12.329982 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.329955    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597bf46569-crsd2_e24d7847-6ea7-46a8-965c-0a07924e0a19/telemeter-client/0.log"
Sep 04 22:26:12.352239 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.352210    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597bf46569-crsd2_e24d7847-6ea7-46a8-965c-0a07924e0a19/reload/0.log"
Sep 04 22:26:12.374596 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.374570    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597bf46569-crsd2_e24d7847-6ea7-46a8-965c-0a07924e0a19/kube-rbac-proxy/0.log"
Sep 04 22:26:12.408149 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.408124    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d58d685d6-d5bg5_ce3cac16-5b9f-4202-a9b2-88470f4f15c3/thanos-query/0.log"
Sep 04 22:26:12.429931 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.429877    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d58d685d6-d5bg5_ce3cac16-5b9f-4202-a9b2-88470f4f15c3/kube-rbac-proxy-web/0.log"
Sep 04 22:26:12.449850 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.449823    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d58d685d6-d5bg5_ce3cac16-5b9f-4202-a9b2-88470f4f15c3/kube-rbac-proxy/0.log"
Sep 04 22:26:12.469482 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.469408    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d58d685d6-d5bg5_ce3cac16-5b9f-4202-a9b2-88470f4f15c3/prom-label-proxy/0.log"
Sep 04 22:26:12.488658 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.488634    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d58d685d6-d5bg5_ce3cac16-5b9f-4202-a9b2-88470f4f15c3/kube-rbac-proxy-rules/0.log"
Sep 04 22:26:12.507726 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:12.507674    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d58d685d6-d5bg5_ce3cac16-5b9f-4202-a9b2-88470f4f15c3/kube-rbac-proxy-metrics/0.log"
Sep 04 22:26:14.983163 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:14.983108    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"]
Sep 04 22:26:14.988390 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:14.988360    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:14.995772 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:14.995733    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"]
Sep 04 22:26:15.025703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.024179    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-lib-modules\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.025703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.024239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-sys\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.025703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.024359    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6hdh\" (UniqueName: \"kubernetes.io/projected/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-kube-api-access-l6hdh\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.025703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.024403    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-proc\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.025703 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.024431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-podres\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125165 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6hdh\" (UniqueName: \"kubernetes.io/projected/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-kube-api-access-l6hdh\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125382 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-proc\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125382 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125206    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-podres\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125382 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125265    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-lib-modules\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125382 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125304    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-sys\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125610 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125470    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-sys\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.125898 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125866    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-proc\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.126018 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.125998    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-podres\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.126120 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.126102    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-lib-modules\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.138483 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.138453    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6hdh\" (UniqueName: \"kubernetes.io/projected/6e3cfc18-6ed2-4385-8783-7e1b94d1afe1-kube-api-access-l6hdh\") pod \"perf-node-gather-daemonset-s8g8w\" (UID: \"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.307994 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.307906    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.611220 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.611174    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"]
Sep 04 22:26:15.752224 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.752186    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w" event={"ID":"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1","Type":"ContainerStarted","Data":"557e3e06a4ae07eb22afc5e4b59b29fc47c62409c163a027994728573d1df65d"}
Sep 04 22:26:15.752503 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.752479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w" event={"ID":"6e3cfc18-6ed2-4385-8783-7e1b94d1afe1","Type":"ContainerStarted","Data":"03ef2ed3d4d603835cb8f7c63f026f76bd033375493ae692c5bf3c502b1653c3"}
Sep 04 22:26:15.752677 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.752663    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:15.767959 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:15.767898    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w" podStartSLOduration=1.76787798 podStartE2EDuration="1.76787798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:26:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:26:15.766786774 +0000 UTC m=+1950.851171619" watchObservedRunningTime="2026-09-04 22:26:15.76787798 +0000 UTC m=+1950.852262828"
Sep 04 22:26:16.763869 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:16.763843    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89dc6_3a03178a-a0f7-4721-9096-725914c8ebd2/dns/0.log"
Sep 04 22:26:16.782686 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:16.782656    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89dc6_3a03178a-a0f7-4721-9096-725914c8ebd2/kube-rbac-proxy/0.log"
Sep 04 22:26:16.908393 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:16.908356    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pj2xr_266ddbfd-33be-4063-8b22-694081563177/dns-node-resolver/0.log"
Sep 04 22:26:17.512829 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:17.512800    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-llxl2_5376eed9-b670-4a76-8ee4-bca13d83d09c/node-ca/0.log"
Sep 04 22:26:18.649542 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:18.649516    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85db8bc976-sc7bd_c7ce2525-b7c3-41a0-82ee-3226a31ff455/kube-auth-proxy/0.log"
Sep 04 22:26:18.712269 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:18.712240    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xqq_441f14f6-59e1-4c21-8441-a8bcffcb193d/payload-pre-processing/0.log"
Sep 04 22:26:19.612246 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:19.612217    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-glw6j_3fd6a430-21d9-4b54-ab7b-a4a1cb9d6194/serve-healthcheck-canary/0.log"
Sep 04 22:26:20.098177 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:20.098145    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-477ql_a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b/kube-rbac-proxy/0.log"
Sep 04 22:26:20.119185 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:20.119158    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-477ql_a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b/exporter/0.log"
Sep 04 22:26:20.137960 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:20.137932    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-477ql_a64ac37b-972f-4bd5-bdbc-be5f5a6ad89b/extractor/0.log"
Sep 04 22:26:21.769714 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:21.769671    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-s8g8w"
Sep 04 22:26:22.568290 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:22.568259    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vvz9x_06fa6853-e605-4a9a-94ca-a3f8d5218f95/manager/0.log"
Sep 04 22:26:22.610431 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:22.610366    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-78b457d45d-7b246_236c059c-3da3-4a9b-883f-f94ed0919cec/manager/0.log"
Sep 04 22:26:22.766698 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:22.766672    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-5hmtg_704c64d7-cdd6-4574-9f9f-90bcb7be3144/manager/0.log"
Sep 04 22:26:22.786358 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:22.786321    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-5hmtg_704c64d7-cdd6-4574-9f9f-90bcb7be3144/copy-manifests/0.log"
Sep 04 22:26:24.347038 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:24.347005    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-587bddd5d4-qzh96_cf36392c-3964-462d-970c-12131932dc58/manager/0.log"
Sep 04 22:26:24.386023 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:24.385993    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-t5w88_c8f5181c-dd72-4add-8068-6e12c5f03a98/openshift-lws-operator/0.log"
Sep 04 22:26:25.762676 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:25.762645    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85db8bc976-sc7bd_c7ce2525-b7c3-41a0-82ee-3226a31ff455/kube-auth-proxy/0.log"
Sep 04 22:26:25.816696 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:25.816662    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xqq_441f14f6-59e1-4c21-8441-a8bcffcb193d/payload-pre-processing/0.log"
Sep 04 22:26:27.598396 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:27.598371    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85db8bc976-sc7bd_c7ce2525-b7c3-41a0-82ee-3226a31ff455/kube-auth-proxy/0.log"
Sep 04 22:26:27.649197 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:27.649170    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xqq_441f14f6-59e1-4c21-8441-a8bcffcb193d/payload-pre-processing/0.log"
Sep 04 22:26:30.973801 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:30.973767    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrw2k_dad701d3-4293-4879-9296-70fcfc8100fd/kube-multus-additional-cni-plugins/0.log"
Sep 04 22:26:30.993661 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:30.993610    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrw2k_dad701d3-4293-4879-9296-70fcfc8100fd/egress-router-binary-copy/0.log"
Sep 04 22:26:31.012596 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:31.012563    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrw2k_dad701d3-4293-4879-9296-70fcfc8100fd/cni-plugins/0.log"
Sep 04 22:26:31.032234 ip-10-0-133-119 kubenswrapper[2565]: I0904 22:26:31.032207    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrw2k_dad701d3-4293-4879-9296-70fcfc8100fd/bond-cni-plugin/0.log"